Systems and methods for retaining source ip in a load balancing mutli-core environment

ABSTRACT

Described herein is a method and system for distributing requests and responses across a multi-core system. Each core executes a packet engine that further processes data packets allocated to that core. A flow distributor executing within the multi-core system forwards client requests to a packet engine on a core that is selected based on a value generated when a hash is applied to a tuple comprising a client IP address, a client port, a server IP address and a server port identified in the request. The packet engine maintains the client IP address, selects a first port of the core, and determines whether a hash of a tuple comprising those values identifies the selected core. A modification is then made to the client request so that the client request includes a tuple comprising the client IP address, the server IP address, the first port and the server port.

RELATED APPLICATION

This application is a continuation of and claims priority to, and thebenefit of, U.S. No-provisional application Ser. No. 12/489,165,entitled “Systems and Methods For Retaining Source IP In A LoadBalancing Multi-Core Environment” and filed on Jun. 22, 2009, which isincorporated herein by reference in its entirety for all purposes.

FIELD OF THE DISCLOSURE

The present application generally relates to data communicationnetworks. In particular, the present application relates to systems andmethods for distributing data packets received by a multi-core system tocores within the multi-core system.

BACKGROUND OF THE DISCLOSURE

In a multi-core system, any one of the cores may be performing the samefunctionality or different functionality. The multi-core system maydeploy a Receive Side Scaler, such as Microsoft's receive side scalingtechnology to distribute packets received from a network interface cardto any core for processing. The Receive Side Scaler may be agnostic tothe functionality being performed on any of the cores. As the ReceiveSide Scaler receives network packets from a network interface card, itforwards the network packet to a core based on a predetermined function.The network packet may be part of a transaction or series of multiplenetwork packets in some context. Some of these network packets may go todifferent cores because of the distribution function of the Receive SideScaler. As such, this may cause challenges in the balance of processingand performing of functionality in the multi-core system.

BRIEF SUMMARY OF THE DISCLOSURE

There exist multi-core systems that can balance network traffic acrossone or more cores in the multi-core system. These multi-core systems canbe included within an appliance or a computing system and can compriseany number of cores, or processors. In some embodiments, multi-coresystems distribute network traffic according to flow distribution modelssuch as functional parallelism, where each core in a multi-core systemis assigned to a different function, or data parallelism where each corein a multi-core system is assigned to a different device or module.These distribution schemes do not take into account the amount ofnetwork traffic, therefore the distribution of network traffic is oftennot even or symmetrical. Thus, there exists a need for a distributionscheme that substantially symmetrically and evenly distributes networktraffic amongst one or more cores in a multi-core system.

In some instances, distribution of network traffic across one or morecores requires changing an attribute of the network traffic to ensurethat return traffic is routed to the originating core. Ensuring symmetrywith regard to the core where a request is transmitted from and the corewhere the response is received, reduces unnecessary copying and cachingof packet data, and provides an even flow of requests and responses toand from the multi-core system. Some systems achieve symmetricaldistribution by changing tuples associated with data packets in thenetwork traffic. The change made to the tuple can be a modification of asource IP address and/or a source port. In some instances, a backendsystem may require that the source IP address and/or source port remainun-modified. In those instances, systems are needed that both maintainthese data packet attributes, and ensure that requests and responses arehandled by substantially the same core in the multi-core system.

Data packets included within network traffic distributed amongst thecores in a multi-core system are sometimes fragmented. In theseinstances, the multi-core system receives data packet fragments ratherthan a whole data packet. Systems are therefore needed that both handledata packet fragments and evenly and symmetrically distribute networktraffic across the cores of a multi-core system.

In one aspect, described herein is an embodiment of a method forproviding symmetrical request and response processing across a packetengine of a plurality of packet engines. Each of the plurality of packetengines executes on a respective core of a plurality of cores in amulti-core system intermediary to a client and a server. A packet engineexecuting on a first core of the multi-core system intermediary to aclient and a server, receives from a flow distributor a request of theclient to the server. The first core is selected by the flow distributorbased on a hash of a first tuple comprising a client internet protocoladdress, a client port, a server internet protocol address and a serverport identified in the client request. The packet engine selects a firstinternet protocol address from the one or more internet protocoladdresses of the first core and a first port from a plurality of portsof the first core. The packet engine then determines that a hash of asecond tuple comprising at least the first internet protocol address andthe first port identifies the first core. The packet engine thenidentifies that the first port is available, and modifies the request ofthe client to identify the first internet protocol address as the clientinternet protocol address and the first port as the client port.

In some embodiments, the packet engine transmits the modified request ofthe client to the server.

The flow distributor, in some embodiments, receives a response from theserver to the request of the client, and distributes the response to thefirst core of the packet engine based on the hash of a third tuplecomprising a client internet protocol address, a client port, a serverinternet protocol address and a server port identified in the response.

In some embodiments, the packet engine determines that the hash of thefirst tuple identifies the first core on which the packet engineexecutes. In other embodiments, the packet engine determines that thehash of the second tuple identifies the first core on which the packetengine executes.

The packet engine, in some embodiments, determines that the first portis not available, selects a second port from the plurality of ports ofthe first core, determines that the second port is available, anddetermines that a hash of a fourth tuple comprising at least the firstinternet protocol address and the second port identifies the first core.The packet engine then modifies the request of the client to identifythe first internet protocol address as the client internet protocoladdress and the second port as the client port.

In one embodiment, the packet engine determines that the first port isnot available, selects a second internet protocol address from the oneor more internet protocol addresses of the first core, selects a secondport from the plurality of ports of the first core, and determines thata hash of a fifth tuple comprising at least the second internet protocoladdress and the second port, identifies the first core. The packetengine then modifies the request of the client to identify the secondinternet protocol address as the client internet protocol address andthe second port as the client port.

The packet engine, in some embodiments, selects a first internetprotocol address from a group of predetermined internet protocoladdresses of the first core. In other embodiments, the packet engineselects a first port from a port table comprising available ports. Eachport, in some embodiments, is selected for inclusion in the port tablebased in part on one or more hashes of local internet protocol addressesof a first core and local ports associated with each local internetprotocol address.

The flow distributor, in many embodiments, executes within themulti-core system. The multi-core system, in some embodiments, comprisesat least two cores, each core storing a port table comprising availableports on that core.

In one embodiment, the first core is selected by the flow distributorbased in part on the hash of the first tuple. In other embodiments, thepacket engine updates a port allocation table to indicate the assignmentof the first port to the data packet.

In some aspects, described herein is a system for providing symmetricalrequest and response processing across a packet engine of a plurality ofpacket engines, each of the plurality of packet engines executing on arespective core of a plurality of cores in a multi-core systemintermediary to a client and a server. The system can comprise amulti-core system intermediary to a client and a server, the multi-coresystem comprising a plurality of cores. Executing within the multi-coresystem can be a flow distributor receiving a request of a client to aserver, and selecting a first core based on a hash of a first tuplecomprising a client internet protocol address, a client port, a serverinternet protocol address and a server port identified in the clientrequest. A packet engine executing on a first core of the multi-coresystem can receive, from the flow distributor, the client request. Thepacket engine can then select a first internet protocol address of oneor more internet protocol addresses of the first core and a first portfrom a plurality of ports of the first core, determine that a hash of asecond tuple comprising at least the first internet protocol address andthe first port, identifies the first core, identify that the first portis available, and modify the client request to identify the firstinternet protocol address as the client internet protocol address andthe first port as the client port.

In another aspect, describe herein is an embodiment of a method fordirecting by a flow distributor network packets to a packet engine of aplurality of packet engines while maintaining a client internet protocoladdress and a client port, each of the plurality of packet enginesexecuting on a core of a plurality of cores in a multi-core systemintermediary to the client and a server. A packet engine executing on afirst core of the multi-core system intermediary to a client and aserver, receives from a flow distributor a client request identifying afirst tuple comprising a client internet protocol address, a clientport, a server internet protocol address and a server port. The flowdistributor selects the first core to receive the client request basedon a hash of the first tuple. The flow distributor further receives aresponse to the client request forwarded to the server by the packetengine, the response generated by the server and comprising a secondtuple identifying, via a hash of the second tuple, a second coredifferent than the first core of the packet engine receiving therequest. The flow distributor forwards the received response to a secondpacket engine of the second core. The flow distributor then directs,responsive to a rule of the flow distributor executing on the secondcore, the response received by the second core to the first core.

In some embodiments forwarding the received response to the secondpacket engine further comprises storing, by the second packet engine ofthe second core, one or more network packets of the response to a memorylocation accessible by the first core. The one or more network packetscan be stored in a shared buffer accessible by each core in themulti-core system.

In another embodiment, a message identifying that the response is to beprocessed by the packet engine of the first core, is sent by a secondcore to the first core.

The second packet engine of the second core, in some embodiments,determines the response corresponds to a request not processed by thesecond packet engine. This determination can further comprisecalculating a hash of a tuple of the response, the hash identifying thefirst core. This determination can also comprise looking up a port in aport allocation table to identify the first core.

In some embodiments, the packet engine on the first core forwards theclient request to a server. When the client request is forwarded, theclient internet protocol address and the client port in the first tuplecan be maintained.

The response, in some embodiments, comprises a second tuple comprisingat least the client internet protocol address and the client port of thefirst tuple. The hash applied to the first tuple, in some embodiments,is substantially the same as the hash applied to the second tuple.

In one embodiment, the flow distributor selects the first core based inpart on a hash of the first tuple.

In some embodiments, the client internet protocol address is maintainedresponsive to a packet engine configured to maintain client internetprotocol addresses. The packet engine, in these embodiments, can beconfigured to maintain client internet protocol addresses responsive toa security policy requiring maintenance of client internet protocoladdresses. In other embodiments, the client port is maintainedresponsive to a packet engine configured to maintain client ports. Thepacket engine, in these embodiments, can be configured to maintain theclient port responsive to a security policy requiring maintenance ofclient ports.

In other aspects, described herein is a method for directing by a flowdistributor fragmented network packets to a packet engine of a pluralityof packet engines, each of the plurality of packet engines executing ona respective core of a plurality of cores in a multi-core systemintermediary to the client and a server. A packet engine executing on afirst core of multi-core system intermediary to a client and a server,receives from a flow distributor a client request identifying a firsttuple comprising a client internet protocol address, a client port, aserver internet protocol address and a server port. The flow distributorcan select the first core to receive the client request based on a hashof the first tuple. The flow distributor can receive a plurality offragments of a response from the server to the request of clientforwarded to the server by the packet engine on the first core. The flowdistributor can then distribute the plurality of fragments of theresponse to a second core responsive to a second hash computed by theflow distributor on the source internet protocol address and destinationinternet protocol address identified by the plurality of fragments. Thesecond packet engine of the second core can then store the plurality offragments and performing one or more fragmentation actions on theplurality of fragments. A determination is then made by a rule of theflow distributor operating on the second core to direct the plurality offragments received by the second core to the first core.

In some embodiments storing the plurality of fragments further comprisesassembling, by the second packet engine, the plurality of fragments.

In other embodiments, determining to direct the plurality of fragmentsto the first core further comprises storing, by the second packetengine, the assembled plurality of fragments in a memory locationaccessible by the first core. In some embodiments the method furthercomprises sending by the second core to the first core a message todirect the first core to process the assembled plurality of fragments.

In some embodiments, determining to direct the plurality of fragments tothe first core further comprises determining by the second core that thefirst core established the connection. In one embodiment, performing afragmentation action further comprises performing an assembly action,while in still other embodiments performing a fragmentation actionfurther comprises performing a bridging action.

The plurality of fragments, in some embodiments, can be steered to thefirst core.

The flow distributor, can in some embodiments, assemble a portion of theplurality of fragments. The flow distributor can then extract the sourceinternet protocol address and the destination internet protocol addressof the second tuple from the portion of the assembled plurality offragments. In other embodiments, the flow distributor assembles theportion of the plurality of fragments until a header of the response isassembled. The flow distributor can then extract the source internetprotocol address and the destination internet protocol address of thesecond tuple from the assembled response header.

In yet another aspect, described herein is an embodiment of a method forproviding symmetrical request and response processing across a packetengine of a plurality of packet engines while maintaining a client'sinternet protocol address and proxying a port for the client, each ofthe plurality of packet engines executing on a core of a plurality ofcores in a multi-core system intermediary to the client and a server. Apacket engine executing on a first core of the multi-core system,intermediary to the client and the server, receives from a flowdistributor a client request identifying a first tuple comprising aclient internet protocol address, a client port, a server internetprotocol address and a server port. The flow distributor forwards therequest to the first core responsive to a first hash of the first tuple.The packet engine can determine to proxy the client port of the requestand maintain the client internet protocol address. The packet engine canalso compute a second hash of the client internet protocol address andthe destination internet protocol address to select a port allocationtable of a plurality of port allocation tables. After selecting the portallocation table, the packet engine can determine that a hash of asecond tuple comprising at least an available first port from theselected port allocation table and the client internet protocol addressidentifies the first core. The packet engine can then modify the clientport of the client request to identify the first port.

In some embodiments, the packet engine transmits the modified clientrequest to the server. The packet engine, in some embodiments, transmitsthe modified client request to a server located at the destinationinternet protocol address. In other embodiments, the packet enginedetermines that the first port of the selected port allocation table isunavailable. Upon making this determination, the packet engine selects asecond port of the selected port allocation table, and determines thesecond port is available. Still further, the packet engine can determinethe first port is unavailable by determining the first port is in use.

In one embodiment, the method further comprises storing a plurality ofport allocation tables on each core in the multi-core system. Each portallocation table can be located at a proxy internet protocol address ofa core on which the port allocation table is stored. The packet enginecan select a port allocation table based in part on a hash of a clientinternet protocol address and a destination address of a first datapacket.

The flow distributor, in some embodiments, receives a first data packetand a second data packet and forwards the first data packet to a firstcore in the multi-core system based in part on a hash of a first tuplecomprising at least a first client internet protocol address and a firstdestination address of the first data packet. The flow distributor thenforwards the second data packet to a second core in the multi-coresystem based in part on a hash of a second tuple comprising at least asecond client internet protocol address and a second destination addressof the second data packet.

In one embodiment, the method further comprises updating the selectedport allocation table to list the first port as unavailable.

In some aspects, described herein is a system for providing symmetricalrequest and response processing across a packet engine of a plurality ofpacket engines while maintaining a client's internet protocol addressand proxying a port for the client, each of the plurality of packetengines executing on a core of a plurality of cores in a multi-coresystem intermediary to the client and a server. The system can comprisea multi-core system intermediary to a client and a server. The systemcan further comprise a flow distributor receiving a request of a clientto a server, and selecting a first core based on a hash of a first tuplecomprising a client internet protocol address, a client port, a serverinternet protocol address and a server port identified in the clientrequest. A packet engine executing on a first core of the multi-coresystem can receive the client request from the flow distributor, anddetermine whether to proxy the client port of the request and maintainthe client internet protocol address. The packet engine then computes asecond hash of the client internet protocol address and the destinationinternet protocol address to select a port allocation table of aplurality of port allocation tables, and determines that a hash of asecond tuple comprising at least an available first port from theselected port allocation table and the client internet protocol address,identifies the first core. The packet engine then modifies the clientport of the client request to identify the first port.

The details of various embodiments of the methods and systems describedherein are set forth in the accompanying drawings and the descriptionbelow.

BRIEF DESCRIPTION OF THE FIGURES

The foregoing and other objects, aspects, features, and advantages ofthe methods and systems described herein will become more apparent andbetter understood by referring to the following description taken inconjunction with the accompanying drawings, in which:

FIG. 1A is a block diagram of an embodiment of a network environment fora client to access a server via an appliance;

FIG. 1B is a block diagram of an embodiment of an environment fordelivering a computing environment from a server to a client via anappliance;

FIG. 1C is a block diagram of another embodiment of an environment fordelivering a computing environment from a server to a client via anappliance;

FIG. 1D is a block diagram of another embodiment of an environment fordelivering a computing environment from a server to a client via anappliance;

FIGS. 1E-1H are block diagrams of embodiments of a computing device;

FIG. 2A is a block diagram of an embodiment of an appliance forprocessing communications between a client and a server;

FIG. 2B is a block diagram of another embodiment of an appliance foroptimizing, accelerating, load-balancing and routing communicationsbetween a client and a server;

FIG. 3 is a block diagram of an embodiment of a client for communicatingwith a server via the appliance;

FIG. 4A is a block diagram of an embodiment of a virtualizationenvironment;

FIG. 4B is a block diagram of another embodiment of a virtualizationenvironment;

FIG. 4C is a block diagram of an embodiment of a virtualized appliance;

FIG. 5A are block diagrams of embodiments of approaches to implementingparallelism in a multi-core system;

FIG. 5B is a block diagram of an embodiment of a system utilizing amulti-core system;

FIG. 5C is a block diagram of another embodiment of an aspect of amulti-core system;

FIG. 6A is a block diagram of an embodiment of a multi-core system;

FIG. 6B is a block diagram of an embodiment of a core within amulti-core system;

FIGS. 7A-7C are flow diagrams of embodiments of a method fordistributing data packets across a multi-core system;

FIG. 8 is a flow diagram of an embodiment of a method for distributingdata packets across a multi-core system based on a hash;

FIG. 9 is a flow diagram of an embodiment of a method for distributingdata packets across a multi-core system via core-to-core messaging;

FIGS. 10A-10B are flow diagrams of embodiments of a method fordistributing data packets across a multi-core system while maintaining aclient IP address and client port;

FIGS. 11A-11B are flow diagrams of embodiments of a method fordistributing data packet fragments across a multi-core system;

FIG. 12A is a flow diagram of an embodiment of a method for distributingdata packets across a multi-core system while maintaining a client IPaddress; and

FIG. 12B is a flow diagram of an embodiment of a method for selecting aport allocation table.

The features and advantages of the methods and systems described hereinwill become more apparent from the detailed description set forth belowwhen taken in conjunction with the drawings, in which like referencecharacters identify corresponding elements throughout. In the drawings,like reference numbers generally indicate identical, functionallysimilar, and/or structurally similar elements.

DETAILED DESCRIPTION OF THE DISCLOSURE

For purposes of reading the description of the various embodimentsbelow, the following descriptions of the sections of the specificationand their respective contents may be helpful:

-   -   Section A describes a network environment and computing        environment which may be useful for practicing embodiments        described herein;    -   Section B describes embodiments of systems and methods for        delivering a computing environment to a remote user;    -   Section C describes embodiments of systems and methods for        accelerating communications between a client and a server;    -   Section D describes embodiments of systems and methods for        virtualizing an application delivery controller;    -   Section E describes embodiments of systems and methods for        providing a multi-core architecture and environment; and    -   Section F describes embodiments of systems and methods for        distributing data packets across a multi-core architecture and        environment.

A. Network and Computing Environment

Prior to discussing the specifics of embodiments of the systems andmethods of an appliance and/or client, it may be helpful to discuss thenetwork and computing environments in which such embodiments may bedeployed. Referring now to FIG. 1A, an embodiment of a networkenvironment is depicted. In brief overview, the network environmentcomprises one or more clients 102 a-102 n (also generally referred to aslocal machine(s) 102, or client(s) 102) in communication with one ormore servers 106 a-106 n (also generally referred to as server(s) 106,or remote machine(s) 106) via one or more networks 104, 104′ (generallyreferred to as network 104). In some embodiments, a client 102communicates with a server 106 via an appliance 200.

Although FIG. 1A shows a network 104 and a network 104′ between theclients 102 and the servers 106, the clients 102 and the servers 106 maybe on the same network 104. The networks 104 and 104′ can be the sametype of network or different types of networks. The network 104 and/orthe network 104′ can be a local-area network (LAN), such as a companyIntranet, a metropolitan area network (MAN), or a wide area network(WAN), such as the Internet or the World Wide Web. In one embodiment,network 104′ may be a private network and network 104 may be a publicnetwork. In some embodiments, network 104 may be a private network andnetwork 104′ a public network. In another embodiment, networks 104 and104′ may both be private networks. In some embodiments, clients 102 maybe located at a branch office of a corporate enterprise communicatingvia a WAN connection over the network 104 to the servers 106 located ata corporate data center.

The network 104 and/or 104′ be any type and/or form of network and mayinclude any of the following: a point to point network, a broadcastnetwork, a wide area network, a local area network, a telecommunicationsnetwork, a data communication network, a computer network, an ATM(Asynchronous Transfer Mode) network, a SONET (Synchronous OpticalNetwork) network, a SDH (Synchronous Digital Hierarchy) network, awireless network and a wireline network. In some embodiments, thenetwork 104 may comprise a wireless link, such as an infrared channel orsatellite band. The topology of the network 104 and/or 104′ may be abus, star, or ring network topology. The network 104 and/or 104′ andnetwork topology may be of any such network or network topology as knownto those ordinarily skilled in the art capable of supporting theoperations described herein.

As shown in FIG. 1A, the appliance 200, which also may be referred to asan interface unit 200 or gateway 200, is shown between the networks 104and 104′. In some embodiments, the appliance 200 may be located onnetwork 104. For example, a branch office of a corporate enterprise maydeploy an appliance 200 at the branch office. In other embodiments, theappliance 200 may be located on network 104′. For example, an appliance200 may be located at a corporate data center. In yet anotherembodiment, a plurality of appliances 200 may be deployed on network104. In some embodiments, a plurality of appliances 200 may be deployedon network 104′. In one embodiment, a first appliance 200 communicateswith a second appliance 200′. In other embodiments, the appliance 200could be a part of any client 102 or server 106 on the same or differentnetwork 104,104′ as the client 102. One or more appliances 200 may belocated at any point in the network or network communications pathbetween a client 102 and a server 106.

In some embodiments, the appliance 200 comprises any of the networkdevices manufactured by Citrix Systems, Inc. of Ft. Lauderdale Fla.,referred to as Citrix NetScaler devices. In other embodiments, theappliance 200 includes any of the product embodiments referred to asWebAccelerator and BigIP manufactured by F5 Networks, Inc. of Seattle,Wash. In another embodiment, the appliance 205 includes any of the DXacceleration device platforms and/or the SSL VPN series of devices, suchas SA 700, SA 2000, SA 4000, and SA 6000 devices manufactured by JuniperNetworks, Inc. of Sunnyvale, Calif. In yet another embodiment, theappliance 200 includes any application acceleration and/or securityrelated appliances and/or software manufactured by Cisco Systems, Inc.of San Jose, Calif., such as the Cisco ACE Application Control EngineModule service software and network modules, and Cisco AVS SeriesApplication Velocity System.

In one embodiment, the system may include multiple, logically-groupedservers 106. In these embodiments, the logical group of servers may bereferred to as a server farm 38. In some of these embodiments, theserves 106 may be geographically dispersed. In some cases, a farm 38 maybe administered as a single entity. In other embodiments, the serverfarm 38 comprises a plurality of server farms 38. In one embodiment, theserver farm executes one or more applications on behalf of one or moreclients 102.

The servers 106 within each farm 38 can be heterogeneous. One or more ofthe servers 106 can operate according to one type of operating systemplatform (e.g., WINDOWS NT, manufactured by Microsoft Corp. of Redmond,Wash.), while one or more of the other servers 106 can operate onaccording to another type of operating system platform (e.g., Unix orLinux). The servers 106 of each farm 38 do not need to be physicallyproximate to another server 106 in the same farm 38. Thus, the group ofservers 106 logically grouped as a farm 38 may be interconnected using awide-area network (WAN) connection or medium-area network (MAN)connection. For example, a farm 38 may include servers 106 physicallylocated in different continents or different regions of a continent,country, state, city, campus, or room. Data transmission speeds betweenservers 106 in the farm 38 can be increased if the servers 106 areconnected using a local-area network (LAN) connection or some form ofdirect connection.

Servers 106 may be referred to as a file server, application server, webserver, proxy server, or gateway server. In some embodiments, a server106 may have the capacity to function as either an application server oras a master application server. In one embodiment, a server 106 mayinclude an Active Directory. The clients 102 may also be referred to asclient nodes or endpoints. In some embodiments, a client 102 has thecapacity to function as both a client node seeking access toapplications on a server and as an application server providing accessto hosted applications for other clients 102 a-102 n. In someembodiments, a client 102 communicates with a server 106. In oneembodiment, the client 102 communicates directly with one of the servers106 in a farm 38.

In another embodiment, the client 102 executes a program neighborhoodapplication to communicate with a server 106 in a farm 38. In stillanother embodiment, the server 106 provides the functionality of amaster node. In some embodiments, the client 102 communicates with theserver 106 in the farm 38 through a network 104. Over the network 104,the client 102 can, for example, request execution of variousapplications hosted by the servers 106 a-106 n in the farm 38 andreceive output of the results of the application execution for display.In some embodiments, only the master node provides the functionalityrequired to identify and provide address information associated with aserver 106′ hosting a requested application.

In one embodiment, the server 106 provides functionality of a webserver. In another embodiment, the server 106 a receives requests fromthe client 102, forwards the requests to a second server 106 b andresponds to the request by the client 102 with a response to the requestfrom the server 106 b. In still another embodiment, the server 106acquires an enumeration of applications available to the client 102 andaddress information associated with a server 106 hosting an applicationidentified by the enumeration of applications. In yet anotherembodiment, the server 106 presents the response to the request to theclient 102 using a web interface. In one embodiment, the client 102communicates directly with the server 106 to access the identifiedapplication. In another embodiment, the client 102 receives applicationoutput data, such as display data, generated by an execution of theidentified application on the server 106.

Referring now to FIG. 1B, an embodiment of a network environmentdeploying multiple appliances 200 is depicted. A first appliance 200 maybe deployed on a first network 104 and a second appliance 200′ on asecond network 104′. For example a corporate enterprise may deploy afirst appliance 200 at a branch office and a second appliance 200′ at adata center. In another embodiment, the first appliance 200 and secondappliance 200′ are deployed on the same network 104 or network 104. Forexample, a first appliance 200 may be deployed for a first server farm38, and a second appliance 200 may be deployed for a second server farm38′. In another example, a first appliance 200 may be deployed at afirst branch office while the second appliance 200′ is deployed at asecond branch office’. In some embodiments, the first appliance 200 andsecond appliance 200′ work in cooperation or in conjunction with eachother to accelerate network traffic or the delivery of application anddata between a client and a server

Referring now to FIG. 1C, another embodiment of a network environmentdeploying the appliance 200 with one or more other types of appliances,such as between one or more WAN optimization appliance 205, 205′ isdepicted. For example a first WAN optimization appliance 205 is shownbetween networks 104 and 104′ and s second WAN optimization appliance205′ may be deployed between the appliance 200 and one or more servers106. By way of example, a corporate enterprise may deploy a first WANoptimization appliance 205 at a branch office and a second WANoptimization appliance 205′ at a data center. In some embodiments, theappliance 205 may be located on network 104′. In other embodiments, theappliance 205′ may be located on network 104. In some embodiments, theappliance 205′ may be located on network 104′ or network 104″. In oneembodiment, the appliance 205 and 205′ are on the same network. Inanother embodiment, the appliance 205 and 205′ are on differentnetworks. In another example, a first WAN optimization appliance 205 maybe deployed for a first server farm 38 and a second WAN optimizationappliance 205′ for a second server farm 38′

In one embodiment, the appliance 205 is a device for accelerating,optimizing or otherwise improving the performance, operation, or qualityof service of any type and form of network traffic, such as traffic toand/or from a WAN connection. In some embodiments, the appliance 205 isa performance enhancing proxy. In other embodiments, the appliance 205is any type and form of WAN optimization or acceleration device,sometimes also referred to as a WAN optimization controller. In oneembodiment, the appliance 205 is any of the product embodiments referredto as WANScaler manufactured by Citrix Systems, Inc. of Ft. Lauderdale,Fla. In other embodiments, the appliance 205 includes any of the productembodiments referred to as BIG-IP link controller and WANjetmanufactured by F5 Networks, Inc. of Seattle, Wash. In anotherembodiment, the appliance 205 includes any of the WX and WXC WANacceleration device platforms manufactured by Juniper Networks, Inc. ofSunnyvale, Calif. In some embodiments, the appliance 205 includes any ofthe steelhead line of WAN optimization appliances manufactured byRiverbed Technology of San Francisco, Calif. In other embodiments, theappliance 205 includes any of the WAN related devices manufactured byExpand Networks Inc. of Roseland, N.J. In one embodiment, the appliance205 includes any of the WAN related appliances manufactured by PacketeerInc. of Cupertino, Calif., such as the PacketShaper, iShared, and SkyXproduct embodiments provided by Packeteer. In yet another embodiment,the appliance 205 includes any WAN related appliances and/or softwaremanufactured by Cisco Systems, Inc. of San Jose, Calif., such as theCisco Wide Area Network Application Services software and networkmodules, and Wide Area Network engine appliances.

In one embodiment, the appliance 205 provides application and dataacceleration services for branch-office or remote offices. In oneembodiment, the appliance 205 includes optimization of Wide Area FileServices (WAFS). In another embodiment, the appliance 205 acceleratesthe delivery of files, such as via the Common Internet File System(CIFS) protocol. In other embodiments, the appliance 205 providescaching in memory and/or storage to accelerate delivery of applicationsand data. In one embodiment, the appliance 205 provides compression ofnetwork traffic at any level of the network stack or at any protocol ornetwork layer. In another embodiment, the appliance 205 providestransport layer protocol optimizations, flow control, performanceenhancements or modifications and/or management to accelerate deliveryof applications and data over a WAN connection. For example, in oneembodiment, the appliance 205 provides Transport Control Protocol (TCP)optimizations. In other embodiments, the appliance 205 providesoptimizations, flow control, performance enhancements or modificationsand/or management for any session or application layer protocol.

In another embodiment, the appliance 205 encoded any type and form ofdata or information into custom or standard TCP and/or IP header fieldsor option fields of network packet to announce presence, functionalityor capability to another appliance 205′. In another embodiment, anappliance 205′ may communicate with another appliance 205′ using dataencoded in both TCP and/or IP header fields or options. For example, theappliance may use TCP option(s) or IP header fields or options tocommunicate one or more parameters to be used by the appliances 205,205′ in performing functionality, such as WAN acceleration, or forworking in conjunction with each other.

In some embodiments, the appliance 200 preserves any of the informationencoded in TCP and/or IP header and/or option fields communicatedbetween appliances 205 and 205′. For example, the appliance 200 mayterminate a transport layer connection traversing the appliance 200,such as a transport layer connection from between a client and a servertraversing appliances 205 and 205′. In one embodiment, the appliance 200identifies and preserves any encoded information in a transport layerpacket transmitted by a first appliance 205 via a first transport layerconnection and communicates a transport layer packet with the encodedinformation to a second appliance 205′ via a second transport layerconnection.

Referring now to FIG. 1D, a network environment for delivering and/oroperating a computing environment on a client 102 is depicted. In someembodiments, a server 106 includes an application delivery system 190for delivering a computing environment or an application and/or datafile to one or more clients 102. In brief overview, a client 10 is incommunication with a server 106 via network 104, 104′ and appliance 200.For example, the client 102 may reside in a remote office of a company,e.g., a branch office, and the server 106 may reside at a corporate datacenter. The client 102 comprises a client agent 120, and a computingenvironment 15. The computing environment 15 may execute or operate anapplication that accesses, processes or uses a data file. The computingenvironment 15, application and/or data file may be delivered via theappliance 200 and/or the server 106.

In some embodiments, the appliance 200 accelerates delivery of acomputing environment 15, or any portion thereof, to a client 102. Inone embodiment, the appliance 200 accelerates the delivery of thecomputing environment 15 by the application delivery system 190. Forexample, the embodiments described herein may be used to acceleratedelivery of a streaming application and data file processable by theapplication from a central corporate data center to a remote userlocation, such as a branch office of the company. In another embodiment,the appliance 200 accelerates transport layer traffic between a client102 and a server 106. The appliance 200 may provide accelerationtechniques for accelerating any transport layer payload from a server106 to a client 102, such as: 1) transport layer connection pooling, 2)transport layer connection multiplexing, 3) transport control protocolbuffering, 4) compression and 5) caching. In some embodiments, theappliance 200 provides load balancing of servers 106 in responding torequests from clients 102. In other embodiments, the appliance 200 actsas a proxy or access server to provide access to the one or more servers106. In another embodiment, the appliance 200 provides a secure virtualprivate network connection from a first network 104 of the client 102 tothe second network 104′ of the server 106, such as an SSL VPNconnection. It yet other embodiments, the appliance 200 providesapplication firewall security, control and management of the connectionand communications between a client 102 and a server 106.

In some embodiments, the application delivery management system 190provides application delivery techniques to deliver a computingenvironment to a desktop of a user, remote or otherwise, based on aplurality of execution methods and based on any authentication andauthorization policies applied via a policy engine 195. With thesetechniques, a remote user may obtain a computing environment and accessto server stored applications and data files from any network connecteddevice 100. In one embodiment, the application delivery system 190 mayreside or execute on a server 106. In another embodiment, theapplication delivery system 190 may reside or execute on a plurality ofservers 106 a-106 n. In some embodiments, the application deliverysystem 190 may execute in a server farm 38. In one embodiment, theserver 106 executing the application delivery system 190 may also storeor provide the application and data file. In another embodiment, a firstset of one or more servers 106 may execute the application deliverysystem 190, and a different server 106 n may store or provide theapplication and data file. In some embodiments, each of the applicationdelivery system 190, the application, and data file may reside or belocated on different servers. In yet another embodiment, any portion ofthe application delivery system 190 may reside, execute or be stored onor distributed to the appliance 200, or a plurality of appliances.

The client 102 may include a computing environment 15 for executing anapplication that uses or processes a data file. The client 102 vianetworks 104, 104′ and appliance 200 may request an application and datafile from the server 106. In one embodiment, the appliance 200 mayforward a request from the client 102 to the server 106. For example,the client 102 may not have the application and data file stored oraccessible locally. In response to the request, the application deliverysystem 190 and/or server 106 may deliver the application and data fileto the client 102. For example, in one embodiment, the server 106 maytransmit the application as an application stream to operate incomputing environment 15 on client 102.

In some embodiments, the application delivery system 190 comprises anyportion of the Citrix Access Suite™ by Citrix Systems, Inc., such as theMetaFrame or Citrix Presentation Server™ and/or any of the Microsoft®Windows Terminal Services manufactured by the Microsoft Corporation. Inone embodiment, the application delivery system 190 may deliver one ormore applications to clients 102 or users via a remote-display protocolor otherwise via remote-based or server-based computing. In anotherembodiment, the application delivery system 190 may deliver one or moreapplications to clients or users via steaming of the application.

In one embodiment, the application delivery system 190 includes a policyengine 195 for controlling and managing the access to, selection ofapplication execution methods and the delivery of applications. In someembodiments, the policy engine 195 determines the one or moreapplications a user or client 102 may access. In another embodiment, thepolicy engine 195 determines how the application should be delivered tothe user or client 102, e.g., the method of execution. In someembodiments, the application delivery system 190 provides a plurality ofdelivery techniques from which to select a method of applicationexecution, such as a server-based computing, streaming or delivering theapplication locally to the client 120 for local execution.

In one embodiment, a client 102 requests execution of an applicationprogram and the application delivery system 190 comprising a server 106selects a method of executing the application program. In someembodiments, the server 106 receives credentials from the client 102. Inanother embodiment, the server 106 receives a request for an enumerationof available applications from the client 102. In one embodiment, inresponse to the request or receipt of credentials, the applicationdelivery system 190 enumerates a plurality of application programsavailable to the client 102. The application delivery system 190receives a request to execute an enumerated application. The applicationdelivery system 190 selects one of a predetermined number of methods forexecuting the enumerated application, for example, responsive to apolicy of a policy engine. The application delivery system 190 mayselect a method of execution of the application enabling the client 102to receive application-output data generated by execution of theapplication program on a server 106. The application delivery system 190may select a method of execution of the application enabling the localmachine 10 to execute the application program locally after retrieving aplurality of application files comprising the application. In yetanother embodiment, the application delivery system 190 may select amethod of execution of the application to stream the application via thenetwork 104 to the client 102.

A client 102 may execute, operate or otherwise provide an application,which can be any type and/or form of software, program, or executableinstructions such as any type and/or form of web browser, web-basedclient, client-server application, a thin-client computing client, anActiveX control, or a Java applet, or any other type and/or form ofexecutable instructions capable of executing on client 102. In someembodiments, the application may be a server-based or a remote-basedapplication executed on behalf of the client 102 on a server 106. In oneembodiments the server 106 may display output to the client 102 usingany thin-client or remote-display protocol, such as the IndependentComputing Architecture (ICA) protocol manufactured by Citrix Systems,Inc. of Ft. Lauderdale, Fla. or the Remote Desktop Protocol (RDP)manufactured by the Microsoft Corporation of Redmond, Wash. Theapplication can use any type of protocol and it can be, for example, anHTTP client, an FTP client, an Oscar client, or a Telnet client. Inother embodiments, the application comprises any type of softwarerelated to VoIP communications, such as a soft IP telephone. In furtherembodiments, the application comprises any application related toreal-time data communications, such as applications for streaming videoand/or audio.

In some embodiments, the server 106 or a server farm 38 may be runningone or more applications, such as an application providing a thin-clientcomputing or remote display presentation application. In one embodiment,the server 106 or server farm 38 executes as an application, any portionof the Citrix Access Suite™ by Citrix Systems, Inc., such as theMetaFrame or Citrix Presentation Server™, and/or any of the Microsoft®Windows Terminal Services manufactured by the Microsoft Corporation. Inone embodiment, the application is an ICA client, developed by CitrixSystems, Inc. of Fort Lauderdale, Fla. In other embodiments, theapplication includes a Remote Desktop (RDP) client, developed byMicrosoft Corporation of Redmond, Wash. Also, the server 106 may run anapplication, which for example, may be an application server providingemail services such as Microsoft Exchange manufactured by the MicrosoftCorporation of Redmond, Wash., a web or Internet server, or a desktopsharing server, or a collaboration server. In some embodiments, any ofthe applications may comprise any type of hosted service or products,such as GoToMeeting™ provided by Citrix Online Division, Inc. of SantaBarbara, Calif., WebEx™ provided by WebEx, Inc. of Santa Clara, Calif.,or Microsoft Office Live Meeting provided by Microsoft Corporation ofRedmond, Wash.

Still referring to FIG. 1D, an embodiment of the network environment mayinclude a monitoring server 106A. The monitoring server 106A may includeany type and form performance monitoring service 198. The performancemonitoring service 198 may include monitoring, measurement and/ormanagement software and/or hardware, including data collection,aggregation, analysis, management and reporting. In one embodiment, theperformance monitoring service 198 includes one or more monitoringagents 197. The monitoring agent 197 includes any software, hardware orcombination thereof for performing monitoring, measurement and datacollection activities on a device, such as a client 102, server 106 oran appliance 200, 205. In some embodiments, the monitoring agent 197includes any type and form of script, such as Visual Basic script, orJavascript. In one embodiment, the monitoring agent 197 executestransparently to any application and/or user of the device. In someembodiments, the monitoring agent 197 is installed and operatedunobtrusively to the application or client. In yet another embodiment,the monitoring agent 197 is installed and operated without anyinstrumentation for the application or device.

In some embodiments, the monitoring agent 197 monitors, measures andcollects data on a predetermined frequency. In other embodiments, themonitoring agent 197 monitors, measures and collects data based upondetection of any type and form of event. For example, the monitoringagent 197 may collect data upon detection of a request for a web page orreceipt of an HTTP response. In another example, the monitoring agent197 may collect data upon detection of any user input events, such as amouse click. The monitoring agent 197 may report or provide anymonitored, measured or collected data to the monitoring service 198. Inone embodiment, the monitoring agent 197 transmits information to themonitoring service 198 according to a schedule or a predeterminedfrequency. In another embodiment, the monitoring agent 197 transmitsinformation to the monitoring service 198 upon detection of an event.

In some embodiments, the monitoring service 198 and/or monitoring agent197 performs monitoring and performance measurement of any networkresource or network infrastructure element, such as a client, server,server farm, appliance 200, appliance 205, or network connection. In oneembodiment, the monitoring service 198 and/or monitoring agent 197performs monitoring and performance measurement of any transport layerconnection, such as a TCP or UDP connection. In another embodiment, themonitoring service 198 and/or monitoring agent 197 monitors and measuresnetwork latency. In yet one embodiment, the monitoring service 198and/or monitoring agent 197 monitors and measures bandwidth utilization.

In other embodiments, the monitoring service 198 and/or monitoring agent197 monitors and measures end-user response times. In some embodiments,the monitoring service 198 performs monitoring and performancemeasurement of an application. In another embodiment, the monitoringservice 198 and/or monitoring agent 197 performs monitoring andperformance measurement of any session or connection to the application.In one embodiment, the monitoring service 198 and/or monitoring agent197 monitors and measures performance of a browser. In anotherembodiment, the monitoring service 198 and/or monitoring agent 197monitors and measures performance of HTTP based transactions. In someembodiments, the monitoring service 198 and/or monitoring agent 197monitors and measures performance of a Voice over IP (VoIP) applicationor session. In other embodiments, the monitoring service 198 and/ormonitoring agent 197 monitors and measures performance of a remotedisplay protocol application, such as an ICA client or RDP client. Inyet another embodiment, the monitoring service 198 and/or monitoringagent 197 monitors and measures performance of any type and form ofstreaming media. In still a further embodiment, the monitoring service198 and/or monitoring agent 197 monitors and measures performance of ahosted application or a Software-As-A-Service (SaaS) delivery model.

In some embodiments, the monitoring service 198 and/or monitoring agent197 performs monitoring and performance measurement of one or moretransactions, requests or responses related to application. In otherembodiments, the monitoring service 198 and/or monitoring agent 197monitors and measures any portion of an application layer stack, such asany .NET or J2EE calls. In one embodiment, the monitoring service 198and/or monitoring agent 197 monitors and measures database or SQLtransactions. In yet another embodiment, the monitoring service 198and/or monitoring agent 197 monitors and measures any method, functionor application programming interface (API) call.

In one embodiment, the monitoring service 198 and/or monitoring agent197 performs monitoring and performance measurement of a delivery ofapplication and/or data from a server to a client via one or moreappliances, such as appliance 200 and/or appliance 205. In someembodiments, the monitoring service 198 and/or monitoring agent 197monitors and measures performance of delivery of a virtualizedapplication. In other embodiments, the monitoring service 198 and/ormonitoring agent 197 monitors and measures performance of delivery of astreaming application. In another embodiment, the monitoring service 198and/or monitoring agent 197 monitors and measures performance ofdelivery of a desktop application to a client and/or the execution ofthe desktop application on the client. In another embodiment, themonitoring service 198 and/or monitoring agent 197 monitors and measuresperformance of a client/server application.

In one embodiment, the monitoring service 198 and/or monitoring agent197 is designed and constructed to provide application performancemanagement for the application delivery system 190. For example, themonitoring service 198 and/or monitoring agent 197 may monitor, measureand manage the performance of the delivery of applications via theCitrix Presentation Server. In this example, the monitoring service 198and/or monitoring agent 197 monitors individual ICA sessions. Themonitoring service 198 and/or monitoring agent 197 may measure the totaland per session system resource usage, as well as application andnetworking performance. The monitoring service 198 and/or monitoringagent 197 may identify the active servers for a given user and/or usersession. In some embodiments, the monitoring service 198 and/ormonitoring agent 197 monitors back-end connections between theapplication delivery system 190 and an application and/or databaseserver. The monitoring service 198 and/or monitoring agent 197 maymeasure network latency, delay and volume per user-session or ICAsession.

In some embodiments, the monitoring service 198 and/or monitoring agent197 measures and monitors memory usage for the application deliverysystem 190, such as total memory usage, per user session and/or perprocess. In other embodiments, the monitoring service 198 and/ormonitoring agent 197 measures and monitors CPU usage the applicationdelivery system 190, such as total CPU usage, per user session and/orper process. In another embodiments, the monitoring service 198 and/ormonitoring agent 197 measures and monitors the time required to log-into an application, a server, or the application delivery system, such asCitrix Presentation Server. In one embodiment, the monitoring service198 and/or monitoring agent 197 measures and monitors the duration auser is logged into an application, a server, or the applicationdelivery system 190. In some embodiments, the monitoring service 198and/or monitoring agent 197 measures and monitors active and inactivesession counts for an application, server or application delivery systemsession. In yet another embodiment, the monitoring service 198 and/ormonitoring agent 197 measures and monitors user session latency.

In yet further embodiments, the monitoring service 198 and/or monitoringagent 197 measures and monitors measures and monitors any type and formof server metrics. In one embodiment, the monitoring service 198 and/ormonitoring agent 197 measures and monitors metrics related to systemmemory, CPU usage, and disk storage. In another embodiment, themonitoring service 198 and/or monitoring agent 197 measures and monitorsmetrics related to page faults, such as page faults per second. In otherembodiments, the monitoring service 198 and/or monitoring agent 197measures and monitors round-trip time metrics. In yet anotherembodiment, the monitoring service 198 and/or monitoring agent 197measures and monitors metrics related to application crashes, errorsand/or hangs.

In some embodiments, the monitoring service 198 and monitoring agent 198includes any of the product embodiments referred to as EdgeSightmanufactured by Citrix Systems, Inc. of Ft. Lauderdale, Fla. In anotherembodiment, the performance monitoring service 198 and/or monitoringagent 198 includes any portion of the product embodiments referred to asthe TrueView product suite manufactured by the Symphoniq Corporation ofPalo Alto, Calif. In one embodiment, the performance monitoring service198 and/or monitoring agent 198 includes any portion of the productembodiments referred to as the TeaLeaf CX product suite manufactured bythe TeaLeaf Technology Inc. of San Francisco, Calif. In otherembodiments, the performance monitoring service 198 and/or monitoringagent 198 includes any portion of the business service managementproducts, such as the BMC Performance Manager and Patrol products,manufactured by BMC Software, Inc. of Houston, Tex.

The client 102, server 106, and appliance 200 may be deployed as and/orexecuted on any type and form of computing device, such as a computer,network device or appliance capable of communicating on any type andform of network and performing the operations described herein. FIGS. 1Eand 1F depict block diagrams of a computing device 100 useful forpracticing an embodiment of the client 102, server 106 or appliance 200.As shown in FIGS. 1E and 1F, each computing device 100 includes acentral processing unit 101, and a main memory unit 122. As shown inFIG. 1E, a computing device 100 may include a visual display device 124,a keyboard 126 and/or a pointing device 127, such as a mouse. Eachcomputing device 100 may also include additional optional elements, suchas one or more input/output devices 130 a-130 b (generally referred tousing reference numeral 130), and a cache memory 140 in communicationwith the central processing unit 101.

The central processing unit 101 is any logic circuitry that responds toand processes instructions fetched from the main memory unit 122. Inmany embodiments, the central processing unit is provided by amicroprocessor unit, such as: those manufactured by Intel Corporation ofMountain View, Calif.; those manufactured by Motorola Corporation ofSchaumburg, Ill.; those manufactured by Transmeta Corporation of SantaClara, Calif.; the RS/6000 processor, those manufactured byInternational Business Machines of White Plains, N.Y.; or thosemanufactured by Advanced Micro Devices of Sunnyvale, Calif. Thecomputing device 100 may be based on any of these processors, or anyother processor capable of operating as described herein.

Main memory unit 122 may be one or more memory chips capable of storingdata and allowing any storage location to be directly accessed by themicroprocessor 101, such as Static random access memory (SRAM), BurstSRAM or SynchBurst SRAM (BSRAM), Dynamic random access memory (DRAM),Fast Page Mode DRAM (FPM DRAM), Enhanced DRAM (EDRAM), Extended DataOutput RAM (EDO RAM), Extended Data Output DRAM (EDO DRAM), BurstExtended Data Output DRAM (BEDO DRAM), Enhanced DRAM (EDRAM),synchronous DRAM (SDRAM), JEDEC SRAM, PC100 SDRAM, Double Data RateSDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), SyncLink DRAM (SLDRAM),Direct Rambus DRAM (DRDRAM), or Ferroelectric RAM (FRAM). The mainmemory 122 may be based on any of the above described memory chips, orany other available memory chips capable of operating as describedherein. In the embodiment shown in FIG. 1E, the processor 101communicates with main memory 122 via a system bus 150 (described inmore detail below). FIG. 1E depicts an embodiment of a computing device100 in which the processor communicates directly with main memory 122via a memory port 103. For example, in FIG. 1F the main memory 122 maybe DRDRAM.

FIG. 1F depicts an embodiment in which the main processor 101communicates directly with cache memory 140 via a secondary bus,sometimes referred to as a backside bus. In other embodiments, the mainprocessor 101 communicates with cache memory 140 using the system bus150. Cache memory 140 typically has a faster response time than mainmemory 122 and is typically provided by SRAM, BSRAM, or EDRAM. In theembodiment shown in FIG. 1E, the processor 101 communicates with variousI/O devices 130 via a local system bus 150. Various busses may be usedto connect the central processing unit 101 to any of the I/O devices130, including a VESA VL bus, an ISA bus, an EISA bus, a MicroChannelArchitecture (MCA) bus, a PCI bus, a PCI-X bus, a PCI-Express bus, or aNuBus. For embodiments in which the I/O device is a video display 124,the processor 101 may use an Advanced Graphics Port (AGP) to communicatewith the display 124. FIG. 1F depicts an embodiment of a computer 100 inwhich the main processor 101 communicates directly with I/O device 130via HyperTransport, Rapid I/O, or InfiniBand. FIG. 1F also depicts anembodiment in which local busses and direct communication are mixed: theprocessor 101 communicates with I/O device 130 using a localinterconnect bus while communicating with I/O device 130 directly.

The computing device 100 may support any suitable installation device116, such as a floppy disk drive for receiving floppy disks such as3.5-inch, 5.25-inch disks or ZIP disks, a CD-ROM drive, a CD-R/RW drive,a DVD-ROM drive, tape drives of various formats, USB device, hard-driveor any other device suitable for installing software and programs suchas any client agent 120, or portion thereof. The computing device 100may further comprise a storage device 128, such as one or more hard diskdrives or redundant arrays of independent disks, for storing anoperating system and other related software, and for storing applicationsoftware programs such as any program related to the client agent 120.Optionally, any of the installation devices 116 could also be used asthe storage device 128. Additionally, the operating system and thesoftware can be run from a bootable medium, for example, a bootable CD,such as KNOPPIX®, a bootable CD for GNU/Linux that is available as aGNU/Linux distribution from knoppix.net.

Furthermore, the computing device 100 may include a network interface118 to interface to a Local Area Network (LAN), Wide Area Network (WAN)or the Internet through a variety of connections including, but notlimited to, standard telephone lines, LAN or WAN links (e.g., 802.11,T1, T3, 56 kb, X.25), broadband connections (e.g., ISDN, Frame Relay,ATM), wireless connections, or some combination of any or all of theabove. The network interface 118 may comprise a built-in networkadapter, network interface card, PCMCIA network card, card bus networkadapter, wireless network adapter, USB network adapter, modem or anyother device suitable for interfacing the computing device 100 to anytype of network capable of communication and performing the operationsdescribed herein. A wide variety of I/O devices 130 a-130 n may bepresent in the computing device 100. Input devices include keyboards,mice, trackpads, trackballs, microphones, and drawing tablets. Outputdevices include video displays, speakers, inkjet printers, laserprinters, and dye-sublimation printers. The I/O devices 130 may becontrolled by an I/O controller 123 as shown in FIG. 1E. The I/Ocontroller may control one or more I/O devices such as a keyboard 126and a pointing device 127, e.g., a mouse or optical pen. Furthermore, anI/O device may also provide storage 128 and/or an installation medium116 for the computing device 100. In still other embodiments, thecomputing device 100 may provide USB connections to receive handheld USBstorage devices such as the USB Flash Drive line of devices manufacturedby Twintech Industry, Inc. of Los Alamitos, California.

In some embodiments, the computing device 100 may comprise or beconnected to multiple display devices 124 a-124 n, which each may be ofthe same or different type and/or form. As such, any of the I/O devices130 a-130 n and/or the I/O controller 123 may comprise any type and/orform of suitable hardware, software, or combination of hardware andsoftware to support, enable or provide for the connection and use ofmultiple display devices 124 a-124 n by the computing device 100. Forexample, the computing device 100 may include any type and/or form ofvideo adapter, video card, driver, and/or library to interface,communicate, connect or otherwise use the display devices 124 a-124 n.In one embodiment, a video adapter may comprise multiple connectors tointerface to multiple display devices 124 a-124 n. In other embodiments,the computing device 100 may include multiple video adapters, with eachvideo adapter connected to one or more of the display devices 124 a-124n. In some embodiments, any portion of the operating system of thecomputing device 100 may be configured for using multiple displays 124a-124 n. In other embodiments, one or more of the display devices 124a-124 n may be provided by one or more other computing devices, such ascomputing devices 100 a and 100 b connected to the computing device 100,for example, via a network. These embodiments may include any type ofsoftware designed and constructed to use another computer's displaydevice as a second display device 124 a for the computing device 100.One ordinarily skilled in the art will recognize and appreciate thevarious ways and embodiments that a computing device 100 may beconfigured to have multiple display devices 124 a-124 n.

In further embodiments, an I/O device 130 may be a bridge 170 betweenthe system bus 150 and an external communication bus, such as a USB bus,an Apple Desktop Bus, an RS-232 serial connection, a SCSI bus, aFireWire bus, a FireWire 800 bus, an Ethernet bus, an AppleTalk bus, aGigabit Ethernet bus, an Asynchronous Transfer Mode bus, a HIPPI bus, aSuper HIPPI bus, a SerialPlus bus, a SCI/LAMP bus, a FibreChannel bus,or a Serial Attached small computer system interface bus.

A computing device 100 of the sort depicted in FIGS. 1E and 1F typicallyoperate under the control of operating systems, which control schedulingof tasks and access to system resources. The computing device 100 can berunning any operating system such as any of the versions of theMicrosoft® Windows operating systems, the different releases of the Unixand Linux operating systems, any version of the Mac OS® for Macintoshcomputers, any embedded operating system, any real-time operatingsystem, any open source operating system, any proprietary operatingsystem, any operating systems for mobile computing devices, or any otheroperating system capable of running on the computing device andperforming the operations described herein. Typical operating systemsinclude: WINDOWS 3.x, WINDOWS 95, WINDOWS 98, WINDOWS 2000, WINDOWS NT3.51, WINDOWS NT 4.0, WINDOWS CE, and WINDOWS XP, all of which aremanufactured by Microsoft Corporation of Redmond, Wash.; MacOS,manufactured by Apple Computer of Cupertino, California; OS/2,manufactured by International Business Machines of Armonk, N.Y.; andLinux, a freely-available operating system distributed by Caldera Corp.of Salt Lake City, Utah, or any type and/or form of a Unix operatingsystem, among others.

In other embodiments, the computing device 100 may have differentprocessors, operating systems, and input devices consistent with thedevice. For example, in one embodiment the computer 100 is a Treo 180,270, 1060, 600 or 650 smart phone manufactured by Palm, Inc. In thisembodiment, the Treo smart phone is operated under the control of thePalmOS operating system and includes a stylus input device as well as afive-way navigator device. Moreover, the computing device 100 can be anyworkstation, desktop computer, laptop or notebook computer, server,handheld computer, mobile telephone, any other computer, or other formof computing or telecommunications device that is capable ofcommunication and that has sufficient processor power and memorycapacity to perform the operations described herein.

As shown in FIG. 1G, the computing device 100 may comprise multipleprocessors and may provide functionality for simultaneous execution ofinstructions or for simultaneous execution of one instruction on morethan one piece of data. In some embodiments, the computing device 100may comprise a parallel processor with one or more cores. In one ofthese embodiments, the computing device 100 is a shared memory paralleldevice, with multiple processors and/or multiple processor cores,accessing all available memory as a single global address space. Inanother of these embodiments, the computing device 100 is a distributedmemory parallel device with multiple processors each accessing localmemory only. In still another of these embodiments, the computing device100 has both some memory which is shared and some memory which can onlybe accessed by particular processors or subsets of processors. In stilleven another of these embodiments, the computing device 100, such as amulti-core microprocessor, combines two or more independent processorsinto a single package, often a single integrated circuit (IC). In yetanother of these embodiments, the computing device 100 includes a chiphaving a CELL BROADBAND ENGINE architecture and including a Powerprocessor element and a plurality of synergistic processing elements,the Power processor element and the plurality of synergistic processingelements linked together by an internal high speed bus, which may bereferred to as an element interconnect bus.

In some embodiments, the processors provide functionality for executionof a single instruction simultaneously on multiple pieces of data(SIMD). In other embodiments, the processors provide functionality forexecution of multiple instructions simultaneously on multiple pieces ofdata (MIMD). In still other embodiments, the processor may use anycombination of SIMD and MIMD cores in a single device.

In some embodiments, the computing device 100 may comprise a graphicsprocessing unit. In one of these embodiments, depicted in FIG. 1H, thecomputing device 100 includes at least one central processing unit 101and at least one graphics processing unit. In another of theseembodiments, the computing device 100 includes at least one parallelprocessing unit and at least one graphics processing unit. In stillanother of these embodiments, the computing device 100 includes aplurality of processing units of any type, one of the plurality ofprocessing units comprising a graphics processing unit.

In some embodiments, a first computing device 100 a executes anapplication on behalf of a user of a client computing device 100 b. Inother embodiments, a computing device 100 a executes a virtual machine,which provides an execution session within which applications execute onbehalf of a user or a client computing devices 100 b. In one of theseembodiments, the execution session is a hosted desktop session. Inanother of these embodiments, the computing device 100 executes aterminal services session. The terminal services session may provide ahosted desktop environment. In still another of these embodiments, theexecution session provides access to a computing environment, which maycomprise one or more of: an application, a plurality of applications, adesktop application, and a desktop session in which one or moreapplications may execute.

B. Appliance Architecture

FIG. 2A illustrates an example embodiment of the appliance 200. Thearchitecture of the appliance 200 in FIG. 2A is provided by way ofillustration only and is not intended to be limiting. As shown in FIG.2, appliance 200 comprises a hardware layer 206 and a software layerdivided into a user space 202 and a kernel space 204.

Hardware layer 206 provides the hardware elements upon which programsand services within kernel space 204 and user space 202 are executed.Hardware layer 206 also provides the structures and elements which allowprograms and services within kernel space 204 and user space 202 tocommunicate data both internally and externally with respect toappliance 200. As shown in FIG. 2, the hardware layer 206 includes aprocessing unit 262 for executing software programs and services, amemory 264 for storing software and data, network ports 266 fortransmitting and receiving data over a network, and an encryptionprocessor 260 for performing functions related to Secure Sockets Layerprocessing of data transmitted and received over the network. In someembodiments, the central processing unit 262 may perform the functionsof the encryption processor 260 in a single processor. Additionally, thehardware layer 206 may comprise multiple processors for each of theprocessing unit 262 and the encryption processor 260. The processor 262may include any of the processors 101 described above in connection withFIGS. 1E and 1F. For example, in one embodiment, the appliance 200comprises a first processor 262 and a second processor 262′. In otherembodiments, the processor 262 or 262′ comprises a multi-core processor.

Although the hardware layer 206 of appliance 200 is generallyillustrated with an encryption processor 260, processor 260 may be aprocessor for performing functions related to any encryption protocol,such as the Secure Socket Layer (SSL) or Transport Layer Security (TLS)protocol. In some embodiments, the processor 260 may be a generalpurpose processor (GPP), and in further embodiments, may have executableinstructions for performing processing of any security related protocol.Although the hardware layer 206 of appliance 200 is illustrated withcertain elements in FIG. 2, the hardware portions or components ofappliance 200 may comprise any type and form of elements, hardware orsoftware, of a computing device, such as the computing device 100illustrated and discussed herein in conjunction with FIGS. 1E and 1F. Insome embodiments, the appliance 200 may comprise a server, gateway,router, switch, bridge or other type of computing or network device, andhave any hardware and/or software elements associated therewith.

The operating system of appliance 200 allocates, manages, or otherwisesegregates the available system memory into kernel space 204 and userspace 204. In example software architecture 200, the operating systemmay be any type and/or form of Unix operating system although themethods and systems described herein are not so limited. As such, theappliance 200 can be running any operating system such as any of theversions of the Microsoft® Windows operating systems, the differentreleases of the Unix and Linux operating systems, any version of the MacOS® for Macintosh computers, any embedded operating system, any networkoperating system, any real-time operating system, any open sourceoperating system, any proprietary operating system, any operatingsystems for mobile computing devices or network devices, or any otheroperating system capable of running on the appliance 200 and performingthe operations described herein.

The kernel space 204 is reserved for running the kernel 230, includingany device drivers, kernel extensions or other kernel related software.As known to those skilled in the art, the kernel 230 is the core of theoperating system, and provides access, control, and management ofresources and hardware-related elements of the application 104. Inaccordance with an embodiment of the appliance 200, the kernel space 204also includes a number of network services or processes working inconjunction with a cache manager 232, sometimes also referred to as theintegrated cache, the benefits of which are described in detail furtherherein. Additionally, the embodiment of the kernel 230 will depend onthe embodiment of the operating system installed, configured, orotherwise used by the device 200.

In one embodiment, the device 200 comprises one network stack 267, suchas a TCP/IP based stack, for communicating with the client 102 and/orthe server 106. In one embodiment, the network stack 267 is used tocommunicate with a first network, such as network 108, and a secondnetwork 110. In some embodiments, the device 200 terminates a firsttransport layer connection, such as a TCP connection of a client 102,and establishes a second transport layer connection to a server 106 foruse by the client 102, e.g., the second transport layer connection isterminated at the appliance 200 and the server 106. The first and secondtransport layer connections may be established via a single networkstack 267. In other embodiments, the device 200 may comprise multiplenetwork stacks, for example 267 and 267′, and the first transport layerconnection may be established or terminated at one network stack 267,and the second transport layer connection on the second network stack267′. For example, one network stack may be for receiving andtransmitting network packet on a first network, and another networkstack for receiving and transmitting network packets on a secondnetwork. In one embodiment, the network stack 267 comprises a buffer 243for queuing one or more network packets for transmission by theappliance 200.

As shown in FIG. 2, the kernel space 204 includes the cache manager 232,a high-speed layer 2-7 integrated packet engine 240, an encryptionengine 234, a policy engine 236 and multi-protocol compression logic238. Running these components or processes 232, 240, 234, 236 and 238 inkernel space 204 or kernel mode instead of the user space 202 improvesthe performance of each of these components, alone and in combination.Kernel operation means that these components or processes 232, 240, 234,236 and 238 run in the core address space of the operating system of thedevice 200. For example, running the encryption engine 234 in kernelmode improves encryption performance by moving encryption and decryptionoperations to the kernel, thereby reducing the number of transitionsbetween the memory space or a kernel thread in kernel mode and thememory space or a thread in user mode. For example, data obtained inkernel mode may not need to be passed or copied to a process or threadrunning in user mode, such as from a kernel level data structure to auser level data structure. In another aspect, the number of contextswitches between kernel mode and user mode are also reduced.Additionally, synchronization of and communications between any of thecomponents or processes 232, 240, 235, 236 and 238 can be performed moreefficiently in the kernel space 204.

In some embodiments, any portion of the components 232, 240, 234, 236and 238 may run or operate in the kernel space 204, while other portionsof these components 232, 240, 234, 236 and 238 may run or operate inuser space 202. In one embodiment, the appliance 200 uses a kernel-leveldata structure providing access to any portion of one or more networkpackets, for example, a network packet comprising a request from aclient 102 or a response from a server 106. In some embodiments, thekernel-level data structure may be obtained by the packet engine 240 viaa transport layer driver interface or filter to the network stack 267.The kernel-level data structure may comprise any interface and/or dataaccessible via the kernel space 204 related to the network stack 267,network traffic or packets received or transmitted by the network stack267. In other embodiments, the kernel-level data structure may be usedby any of the components or processes 232, 240, 234, 236 and 238 toperform the desired operation of the component or process. In oneembodiment, a component 232, 240, 234, 236 and 238 is running in kernelmode 204 when using the kernel-level data structure, while in anotherembodiment, the component 232, 240, 234, 236 and 238 is running in usermode when using the kernel-level data structure. In some embodiments,the kernel-level data structure may be copied or passed to a secondkernel-level data structure, or any desired user-level data structure.

The cache manager 232 may comprise software, hardware or any combinationof software and hardware to provide cache access, control and managementof any type and form of content, such as objects or dynamicallygenerated objects served by the originating servers 106. The data,objects or content processed and stored by the cache manager 232 maycomprise data in any format, such as a markup language, or communicatedvia any protocol. In some embodiments, the cache manager 232 duplicatesoriginal data stored elsewhere or data previously computed, generated ortransmitted, in which the original data may require longer access timeto fetch, compute or otherwise obtain relative to reading a cache memoryelement. Once the data is stored in the cache memory element, future usecan be made by accessing the cached copy rather than refetching orrecomputing the original data, thereby reducing the access time. In someembodiments, the cache memory element may comprise a data object inmemory 264 of device 200. In other embodiments, the cache memory elementmay comprise memory having a faster access time than memory 264. Inanother embodiment, the cache memory element may comprise any type andform of storage element of the device 200, such as a portion of a harddisk. In some embodiments, the processing unit 262 may provide cachememory for use by the cache manager 232. In yet further embodiments, thecache manager 232 may use any portion and combination of memory,storage, or the processing unit for caching data, objects, and othercontent.

Furthermore, the cache manager 232 includes any logic, functions, rules,or operations to perform any embodiments of the techniques of theappliance 200 described herein. For example, the cache manager 232includes logic or functionality to invalidate objects based on theexpiration of an invalidation time period or upon receipt of aninvalidation command from a client 102 or server 106. In someembodiments, the cache manager 232 may operate as a program, service,process or task executing in the kernel space 204, and in otherembodiments, in the user space 202. In one embodiment, a first portionof the cache manager 232 executes in the user space 202 while a secondportion executes in the kernel space 204. In some embodiments, the cachemanager 232 can comprise any type of general purpose processor (GPP), orany other type of integrated circuit, such as a Field Programmable GateArray (FPGA), Programmable Logic Device (PLD), or Application SpecificIntegrated Circuit (ASIC).

The policy engine 236 may include, for example, an intelligentstatistical engine or other programmable application(s). In oneembodiment, the policy engine 236 provides a configuration mechanism toallow a user to identify, specify, define or configure a caching policy.Policy engine 236, in some embodiments, also has access to memory tosupport data structures such as lookup tables or hash tables to enableuser-selected caching policy decisions. In other embodiments, the policyengine 236 may comprise any logic, rules, functions or operations todetermine and provide access, control and management of objects, data orcontent being cached by the appliance 200 in addition to access, controland management of security, network traffic, network access, compressionor any other function or operation performed by the appliance 200.Further examples of specific caching policies are further describedherein.

The encryption engine 234 comprises any logic, business rules, functionsor operations for handling the processing of any security relatedprotocol, such as SSL or TLS, or any function related thereto. Forexample, the encryption engine 234 encrypts and decrypts networkpackets, or any portion thereof, communicated via the appliance 200. Theencryption engine 234 may also setup or establish SSL or TLS connectionson behalf of the client 102 a-102 n, server 106 a-106 n, or appliance200. As such, the encryption engine 234 provides offloading andacceleration of SSL processing. In one embodiment, the encryption engine234 uses a tunneling protocol to provide a virtual private networkbetween a client 102 a-102 n and a server 106 a-106 n. In someembodiments, the encryption engine 234 is in communication with theEncryption processor 260. In other embodiments, the encryption engine234 comprises executable instructions running on the Encryptionprocessor 260.

The multi-protocol compression engine 238 comprises any logic, businessrules, function or operations for compressing one or more protocols of anetwork packet, such as any of the protocols used by the network stack267 of the device 200. In one embodiment, multi-protocol compressionengine 238 compresses bi-directionally between clients 102 a-102 n andservers 106 a-106 n any TCP/IP based protocol, including MessagingApplication Programming Interface (MAPI) (email), File Transfer Protocol(FTP), HyperText Transfer Protocol (HTTP), Common Internet File System(CIFS) protocol (file transfer), Independent Computing Architecture(ICA) protocol, Remote Desktop Protocol (RDP), Wireless ApplicationProtocol (WAP), Mobile IP protocol, and Voice Over IP (VoIP) protocol.In other embodiments, multi-protocol compression engine 238 providescompression of Hypertext Markup Language (HTML) based protocols and insome embodiments, provides compression of any markup languages, such asthe Extensible Markup Language (XML). In one embodiment, themulti-protocol compression engine 238 provides compression of anyhigh-performance protocol, such as any protocol designed for appliance200 to appliance 200 communications. In another embodiment, themulti-protocol compression engine 238 compresses any payload of or anycommunication using a modified transport control protocol, such asTransaction TCP (T/TCP), TCP with selection acknowledgements (TCP-SACK),TCP with large windows (TCP-LW), a congestion prediction protocol suchas the TCP-Vegas protocol, and a TCP spoofing protocol.

As such, the multi-protocol compression engine 238 acceleratesperformance for users accessing applications via desktop clients, e.g.,Microsoft Outlook and non-Web thin clients, such as any client launchedby popular enterprise applications like Oracle, SAP and Siebel, and evenmobile clients, such as the Pocket PC. In some embodiments, themulti-protocol compression engine 238 by executing in the kernel mode204 and integrating with packet processing engine 240 accessing thenetwork stack 267 is able to compress any of the protocols carried bythe TCP/IP protocol, such as any application layer protocol.

High speed layer 2-7 integrated packet engine 240, also generallyreferred to as a packet processing engine or packet engine, isresponsible for managing the kernel-level processing of packets receivedand transmitted by appliance 200 via network ports 266. The high speedlayer 2-7 integrated packet engine 240 may comprise a buffer for queuingone or more network packets during processing, such as for receipt of anetwork packet or transmission of a network packet. Additionally, thehigh speed layer 2-7 integrated packet engine 240 is in communicationwith one or more network stacks 267 to send and receive network packetsvia network ports 266. The high speed layer 2-7 integrated packet engine240 works in conjunction with encryption engine 234, cache manager 232,policy engine 236 and multi-protocol compression logic 238. Inparticular, encryption engine 234 is configured to perform SSLprocessing of packets, policy engine 236 is configured to performfunctions related to traffic management such as request-level contentswitching and request-level cache redirection, and multi-protocolcompression logic 238 is configured to perform functions related tocompression and decompression of data.

The high speed layer 2-7 integrated packet engine 240 includes a packetprocessing timer 242. In one embodiment, the packet processing timer 242provides one or more time intervals to trigger the processing ofincoming, i.e., received, or outgoing, i.e., transmitted, networkpackets. In some embodiments, the high speed layer 2-7 integrated packetengine 240 processes network packets responsive to the timer 242. Thepacket processing timer 242 provides any type and form of signal to thepacket engine 240 to notify, trigger, or communicate a time relatedevent, interval or occurrence. In many embodiments, the packetprocessing timer 242 operates in the order of milliseconds, such as forexample 100 ms, 50 ms or 25 ms. For example, in some embodiments, thepacket processing timer 242 provides time intervals or otherwise causesa network packet to be processed by the high speed layer 2-7 integratedpacket engine 240 at a 10 ms time interval, while in other embodiments,at a 5 ms time interval, and still yet in further embodiments, as shortas a 3, 2, or 1 ms time interval. The high speed layer 2-7 integratedpacket engine 240 may be interfaced, integrated or in communication withthe encryption engine 234, cache manager 232, policy engine 236 andmulti-protocol compression engine 238 during operation. As such, any ofthe logic, functions, or operations of the encryption engine 234, cachemanager 232, policy engine 236 and multi-protocol compression logic 238may be performed responsive to the packet processing timer 242 and/orthe packet engine 240. Therefore, any of the logic, functions, oroperations of the encryption engine 234, cache manager 232, policyengine 236 and multi-protocol compression logic 238 may be performed atthe granularity of time intervals provided via the packet processingtimer 242, for example, at a time interval of less than or equal to 10ms. For example, in one embodiment, the cache manager 232 may performinvalidation of any cached objects responsive to the high speed layer2-7 integrated packet engine 240 and/or the packet processing timer 242.In another embodiment, the expiry or invalidation time of a cachedobject can be set to the same order of granularity as the time intervalof the packet processing timer 242, such as at every 10 ms.

In contrast to kernel space 204, user space 202 is the memory area orportion of the operating system used by user mode applications orprograms otherwise running in user mode. A user mode application may notaccess kernel space 204 directly and uses service calls in order toaccess kernel services. As shown in FIG. 2, user space 202 of appliance200 includes a graphical user interface (GUI) 210, a command lineinterface (CLI) 212, shell services 214, health monitoring program 216,and daemon services 218. GUI 210 and CLI 212 provide a means by which asystem administrator or other user can interact with and control theoperation of appliance 200, such as via the operating system of theappliance 200. The GUI 210 or CLI 212 can comprise code running in userspace 202 or kernel space 204. The GUI 210 may be any type and form ofgraphical user interface and may be presented via text, graphical orotherwise, by any type of program or application, such as a browser. TheCLI 212 may be any type and form of command line or text-basedinterface, such as a command line provided by the operating system. Forexample, the CLI 212 may comprise a shell, which is a tool to enableusers to interact with the operating system. In some embodiments, theCLI 212 may be provided via a bash, csh, tcsh, or ksh type shell. Theshell services 214 comprises the programs, services, tasks, processes orexecutable instructions to support interaction with the appliance 200 oroperating system by a user via the GUI 210 and/or CLI 212.

Health monitoring program 216 is used to monitor, check, report andensure that network systems are functioning properly and that users arereceiving requested content over a network. Health monitoring program216 comprises one or more programs, services, tasks, processes orexecutable instructions to provide logic, rules, functions or operationsfor monitoring any activity of the appliance 200. In some embodiments,the health monitoring program 216 intercepts and inspects any networktraffic passed via the appliance 200. In other embodiments, the healthmonitoring program 216 interfaces by any suitable means and/ormechanisms with one or more of the following: the encryption engine 234,cache manager 232, policy engine 236, multi-protocol compression logic238, packet engine 240, daemon services 218, and shell services 214. Assuch, the health monitoring program 216 may call any applicationprogramming interface (API) to determine a state, status, or health ofany portion of the appliance 200. For example, the health monitoringprogram 216 may ping or send a status inquiry on a periodic basis tocheck if a program, process, service or task is active and currentlyrunning. In another example, the health monitoring program 216 may checkany status, error or history logs provided by any program, process,service or task to determine any condition, status or error with anyportion of the appliance 200.

Daemon services 218 are programs that run continuously or in thebackground and handle periodic service requests received by appliance200. In some embodiments, a daemon service may forward the requests toother programs or processes, such as another daemon service 218 asappropriate. As known to those skilled in the art, a daemon service 218may run unattended to perform continuous or periodic system widefunctions, such as network control, or to perform any desired task. Insome embodiments, one or more daemon services 218 run in the user space202, while in other embodiments, one or more daemon services 218 run inthe kernel space.

Referring now to FIG. 2B, another embodiment of the appliance 200 isdepicted. In brief overview, the appliance 200 provides one or more ofthe following services, functionality or operations: SSL VPNconnectivity 280, switching/load balancing 284, Domain Name Serviceresolution 286, acceleration 288 and an application firewall 290 forcommunications between one or more clients 102 and one or more servers106. Each of the servers 106 may provide one or more network relatedservices 270 a-270 n (referred to as services 270). For example, aserver 106 may provide an http service 270. The appliance 200 comprisesone or more virtual servers or virtual internet protocol servers,referred to as a vServer, VIP server, or just VIP 275 a-275 n (alsoreferred herein as vServer 275). The vServer 275 receives, intercepts orotherwise processes communications between a client 102 and a server 106in accordance with the configuration and operations of the appliance200.

The vServer 275 may comprise software, hardware or any combination ofsoftware and hardware. The vServer 275 may comprise any type and form ofprogram, service, task, process or executable instructions operating inuser mode 202, kernel mode 204 or any combination thereof in theappliance 200. The vServer 275 includes any logic, functions, rules, oroperations to perform any embodiments of the techniques describedherein, such as SSL VPN 280, switching/load balancing 284, Domain NameService resolution 286, acceleration 288 and an application firewall290. In some embodiments, the vServer 275 establishes a connection to aservice 270 of a server 106. The service 275 may comprise any program,application, process, task or set of executable instructions capable ofconnecting to and communicating to the appliance 200, client 102 orvServer 275. For example, the service 275 may comprise a web server,http server, ftp, email or database server. In some embodiments, theservice 270 is a daemon process or network driver for listening,receiving and/or sending communications for an application, such asemail, database or an enterprise application. In some embodiments, theservice 270 may communicate on a specific IP address, or IP address andport.

In some embodiments, the vServer 275 applies one or more policies of thepolicy engine 236 to network communications between the client 102 andserver 106. In one embodiment, the policies are associated with aVServer 275. In another embodiment, the policies are based on a user, ora group of users. In yet another embodiment, a policy is global andapplies to one or more vServers 275 a-275 n, and any user or group ofusers communicating via the appliance 200. In some embodiments, thepolicies of the policy engine have conditions upon which the policy isapplied based on any content of the communication, such as internetprotocol address, port, protocol type, header or fields in a packet, orthe context of the communication, such as user, group of the user,vServer 275, transport layer connection, and/or identification orattributes of the client 102 or server 106.

In other embodiments, the appliance 200 communicates or interfaces withthe policy engine 236 to determine authentication and/or authorizationof a remote user or a remote client 102 to access the computingenvironment 15, application, and/or data file from a server 106. Inanother embodiment, the appliance 200 communicates or interfaces withthe policy engine 236 to determine authentication and/or authorizationof a remote user or a remote client 102 to have the application deliverysystem 190 deliver one or more of the computing environment 15,application, and/or data file. In yet another embodiment, the appliance200 establishes a VPN or SSL VPN connection based on the policy engine's236 authentication and/or authorization of a remote user or a remoteclient 102 In one embodiment, the appliance 200 controls the flow ofnetwork traffic and communication sessions based on policies of thepolicy engine 236. For example, the appliance 200 may control the accessto a computing environment 15, application or data file based on thepolicy engine 236.

In some embodiments, the vServer 275 establishes a transport layerconnection, such as a TCP or UDP connection with a client 102 via theclient agent 120. In one embodiment, the vServer 275 listens for andreceives communications from the client 102. In other embodiments, thevServer 275 establishes a transport layer connection, such as a TCP orUDP connection with a client server 106. In one embodiment, the vServer275 establishes the transport layer connection to an internet protocoladdress and port of a server 270 running on the server 106. In anotherembodiment, the vServer 275 associates a first transport layerconnection to a client 102 with a second transport layer connection tothe server 106. In some embodiments, a vServer 275 establishes a pool oftransport layer connections to a server 106 and multiplexes clientrequests via the pooled transport layer connections.

In some embodiments, the appliance 200 provides a SSL VPN connection 280between a client 102 and a server 106. For example, a client 102 on afirst network 102 requests to establish a connection to a server 106 ona second network 104′. In some embodiments, the second network 104′ isnot routable from the first network 104. In other embodiments, theclient 102 is on a public network 104 and the server 106 is on a privatenetwork 104′, such as a corporate network. In one embodiment, the clientagent 120 intercepts communications of the client 102 on the firstnetwork 104, encrypts the communications, and transmits thecommunications via a first transport layer connection to the appliance200. The appliance 200 associates the first transport layer connectionon the first network 104 to a second transport layer connection to theserver 106 on the second network 104. The appliance 200 receives theintercepted communication from the client agent 102, decrypts thecommunications, and transmits the communication to the server 106 on thesecond network 104 via the second transport layer connection. The secondtransport layer connection may be a pooled transport layer connection.As such, the appliance 200 provides an end-to-end secure transport layerconnection for the client 102 between the two networks 104, 104′.

In one embodiment, the appliance 200 hosts an intranet internet protocolor intranetIP 282 address of the client 102 on the virtual privatenetwork 104. The client 102 has a local network identifier, such as aninternet protocol (IP) address and/or host name on the first network104. When connected to the second network 104′ via the appliance 200,the appliance 200 establishes, assigns or otherwise provides anIntranetIP, which is network identifier, such as IP address and/or hostname, for the client 102 on the second network 104′. The appliance 200listens for and receives on the second or private network 104′ for anycommunications directed towards the client 102 using the client'sestablished IntranetIP 282. In one embodiment, the appliance 200 acts asor on behalf of the client 102 on the second private network 104. Forexample, in another embodiment, a vServer 275 listens for and respondsto communications to the IntranetIP 282 of the client 102. In someembodiments, if a computing device 100 on the second network 104′transmits a request, the appliance 200 processes the request as if itwere the client 102. For example, the appliance 200 may respond to aping to the client's IntranetIP 282. In another example, the appliancemay establish a connection, such as a TCP or UDP connection, withcomputing device 100 on the second network 104 requesting a connectionwith the client's IntranetIP 282.

In some embodiments, the appliance 200 provides one or more of thefollowing acceleration techniques 288 to communications between theclient 102 and server 106: 1) compression; 2) decompression; 3)Transmission Control Protocol pooling; 4) Transmission Control Protocolmultiplexing; 5) Transmission Control Protocol buffering; and 6)caching. In one embodiment, the appliance 200 relieves servers 106 ofmuch of the processing load caused by repeatedly opening and closingtransport layers connections to clients 102 by opening one or moretransport layer connections with each server 106 and maintaining theseconnections to allow repeated data accesses by clients via the Internet.This technique is referred to herein as “connection pooling”.

In some embodiments, in order to seamlessly splice communications from aclient 102 to a server 106 via a pooled transport layer connection, theappliance 200 translates or multiplexes communications by modifyingsequence number and acknowledgment numbers at the transport layerprotocol level. This is referred to as “connection multiplexing”. Insome embodiments, no application layer protocol interaction is required.For example, in the case of an in-bound packet (that is, a packetreceived from a client 102), the source network address of the packet ischanged to that of an output port of appliance 200, and the destinationnetwork address is changed to that of the intended server. In the caseof an outbound packet (that is, one received from a server 106), thesource network address is changed from that of the server 106 to that ofan output port of appliance 200 and the destination address is changedfrom that of appliance 200 to that of the requesting client 102. Thesequence numbers and acknowledgment numbers of the packet are alsotranslated to sequence numbers and acknowledgement expected by theclient 102 on the appliance's 200 transport layer connection to theclient 102. In some embodiments, the packet checksum of the transportlayer protocol is recalculated to account for these translations.

In another embodiment, the appliance 200 provides switching orload-balancing functionality 284 for communications between the client102 and server 106. In some embodiments, the appliance 200 distributestraffic and directs client requests to a server 106 based on layer 4 orapplication-layer request data. In one embodiment, although the networklayer or layer 2 of the network packet identifies a destination server106, the appliance 200 determines the server 106 to distribute thenetwork packet by application information and data carried as payload ofthe transport layer packet. In one embodiment, the health monitoringprograms 216 of the appliance 200 monitor the health of servers todetermine the server 106 for which to distribute a client's request. Insome embodiments, if the appliance 200 detects a server 106 is notavailable or has a load over a predetermined threshold, the appliance200 can direct or distribute client requests to another server 106.

In some embodiments, the appliance 200 acts as a Domain Name Service(DNS) resolver or otherwise provides resolution of a DNS request fromclients 102. In some embodiments, the appliance intercepts' a DNSrequest transmitted by the client 102. In one embodiment, the appliance200 responds to a client's DNS request with an IP address of or hostedby the appliance 200. In this embodiment, the client 102 transmitsnetwork communication for the domain name to the appliance 200. Inanother embodiment, the appliance 200 responds to a client's DNS requestwith an IP address of or hosted by a second appliance 200′. In someembodiments, the appliance 200 responds to a client's DNS request withan IP address of a server 106 determined by the appliance 200.

In yet another embodiment, the appliance 200 provides applicationfirewall functionality 290 for communications between the client 102 andserver 106. In one embodiment, the policy engine 236 provides rules fordetecting and blocking illegitimate requests. In some embodiments, theapplication firewall 290 protects against denial of service (DoS)attacks. In other embodiments, the appliance inspects the content ofintercepted requests to identify and block application-based attacks. Insome embodiments, the rules/policy engine 236 comprises one or moreapplication firewall or security control policies for providingprotections against various classes and types of web or Internet basedvulnerabilities, such as one or more of the following: 1) bufferoverflow, 2) CGI-BIN parameter manipulation, 3) form/hidden fieldmanipulation, 4) forceful browsing, 5) cookie or session poisoning, 6)broken access control list (ACLs) or weak passwords, 7) cross-sitescripting (XSS), 8) command injection, 9) SQL injection, 10) errortriggering sensitive information leak, 11) insecure use of cryptography,12) server misconfiguration, 13) back doors and debug options, 14)website defacement, 15) platform or operating systems vulnerabilities,and 16) zero-day exploits. In an embodiment, the application firewall290 provides HTML form field protection in the form of inspecting oranalyzing the network communication for one or more of the following: 1)required fields are returned, 2) no added field allowed, 3) read-onlyand hidden field enforcement, 4) drop-down list and radio button fieldconformance, and 5) form-field max-length enforcement. In someembodiments, the application firewall 290 ensures cookies are notmodified. In other embodiments, the application firewall 290 protectsagainst forceful browsing by enforcing legal URLs.

In still yet other embodiments, the application firewall 290 protectsany confidential information contained in the network communication. Theapplication firewall 290 may inspect or analyze any networkcommunication in accordance with the rules or polices of the engine 236to identify any confidential information in any field of the networkpacket. In some embodiments, the application firewall 290 identifies inthe network communication one or more occurrences of a credit cardnumber, password, social security number, name, patient code, contactinformation, and age. The encoded portion of the network communicationmay comprise these occurrences or the confidential information. Based onthese occurrences, in one embodiment, the application firewall 290 maytake a policy action on the network communication, such as preventtransmission of the network communication. In another embodiment, theapplication firewall 290 may rewrite, remove or otherwise mask suchidentified occurrence or confidential information.

Still referring to FIG. 2B, the appliance 200 may include a performancemonitoring agent 197 as discussed above in conjunction with FIG. 1D. Inone embodiment, the appliance 200 receives the monitoring agent 197 fromthe monitoring service 198 or monitoring server 106 as depicted in FIG.1D. In some embodiments, the appliance 200 stores the monitoring agent197 in storage, such as disk, for delivery to any client or server incommunication with the appliance 200. For example, in one embodiment,the appliance 200 transmits the monitoring agent 197 to a client uponreceiving a request to establish a transport layer connection. In otherembodiments, the appliance 200 transmits the monitoring agent 197 uponestablishing the transport layer connection with the client 102. Inanother embodiment, the appliance 200 transmits the monitoring agent 197to the client upon intercepting or detecting a request for a web page.In yet another embodiment, the appliance 200 transmits the monitoringagent 197 to a client or a server in response to a request from themonitoring server 198. In one embodiment, the appliance 200 transmitsthe monitoring agent 197 to a second appliance 200′ or appliance 205.

In other embodiments, the appliance 200 executes the monitoring agent197. In one embodiment, the monitoring agent 197 measures and monitorsthe performance of any application, program, process, service, task orthread executing on the appliance 200. For example, the monitoring agent197 may monitor and measure performance and operation of vServers275A-275N. In another embodiment, the monitoring agent 197 measures andmonitors the performance of any transport layer connections of theappliance 200. In some embodiments, the monitoring agent 197 measuresand monitors the performance of any user sessions traversing theappliance 200. In one embodiment, the monitoring agent 197 measures andmonitors the performance of any virtual private network connectionsand/or sessions traversing the appliance 200, such an SSL VPN session.In still further embodiments, the monitoring agent 197 measures andmonitors the memory, CPU and disk usage and performance of the appliance200. In yet another embodiment, the monitoring agent 197 measures andmonitors the performance of any acceleration technique 288 performed bythe appliance 200, such as SSL offloading, connection pooling andmultiplexing, caching, and compression. In some embodiments, themonitoring agent 197 measures and monitors the performance of any loadbalancing and/or content switching 284 performed by the appliance 200.In other embodiments, the monitoring agent 197 measures and monitors theperformance of application firewall 290 protection and processingperformed by the appliance 200.

C. Client Agent

Referring now to FIG. 3, an embodiment of the client agent 120 isdepicted. The client 102 includes a client agent 120 for establishingand exchanging communications with the appliance 200 and/or server 106via a network 104. In brief overview, the client 102 operates oncomputing device 100 having an operating system with a kernel mode 302and a user mode 303, and a network stack 310 with one or more layers 310a-310 b. The client 102 may have installed and/or execute one or moreapplications. In some embodiments, one or more applications maycommunicate via the network stack 310 to a network 104. One of theapplications, such as a web browser, may also include a first program322. For example, the first program 322 may be used in some embodimentsto install and/or execute the client agent 120, or any portion thereof.The client agent 120 includes an interception mechanism, or interceptor350, for intercepting network communications from the network stack 310from the one or more applications.

The network stack 310 of the client 102 may comprise any type and formof software, or hardware, or any combinations thereof, for providingconnectivity to and communications with a network. In one embodiment,the network stack 310 comprises a software implementation for a networkprotocol suite. The network stack 310 may comprise one or more networklayers, such as any networks layers of the Open Systems Interconnection(OSI) communications model as those skilled in the art recognize andappreciate. As such, the network stack 310 may comprise any type andform of protocols for any of the following layers of the OSI model: 1)physical link layer, 2) data link layer, 3) network layer, 4) transportlayer, 5) session layer, 6) presentation layer, and 7) applicationlayer. In one embodiment, the network stack 310 may comprise a transportcontrol protocol (TCP) over the network layer protocol of the internetprotocol (IP), generally referred to as TCP/IP. In some embodiments, theTCP/IP protocol may be carried over the Ethernet protocol, which maycomprise any of the family of IEEE wide-area-network (WAN) orlocal-area-network (LAN) protocols, such as those protocols covered bythe IEEE 802.3. In some embodiments, the network stack 310 comprises anytype and form of a wireless protocol, such as IEEE 802.11 and/or mobileinternet protocol.

In view of a TCP/IP based network, any TCP/IP based protocol may beused, including Messaging Application Programming Interface (MAPI)(email), File Transfer Protocol (FTP), HyperText Transfer Protocol(HTTP), Common Internet File System (CIFS) protocol (file transfer),Independent Computing Architecture (ICA) protocol, Remote DesktopProtocol (RDP), Wireless Application Protocol (WAP), Mobile IP protocol,and Voice Over IP (VoIP) protocol. In another embodiment, the networkstack 310 comprises any type and form of transport control protocol,such as a modified transport control protocol, for example a TransactionTCP (T/TCP), TCP with selection acknowledgements (TCP-SACK), TCP withlarge windows (TCP-LW), a congestion prediction protocol such as theTCP-Vegas protocol, and a TCP spoofing protocol. In other embodiments,any type and form of user datagram protocol (UDP), such as UDP over IP,may be used by the network stack 310, such as for voice communicationsor real-time data communications.

Furthermore, the network stack 310 may include one or more networkdrivers supporting the one or more layers, such as a TCP driver or anetwork layer driver. The network drivers may be included as part of theoperating system of the computing device 100 or as part of any networkinterface cards or other network access components of the computingdevice 100. In some embodiments, any of the network drivers of thenetwork stack 310 may be customized, modified or adapted to provide acustom or modified portion of the network stack 310 in support of any ofthe techniques described herein. In other embodiments, the accelerationprogram 120 is designed and constructed to operate with or work inconjunction with the network stack 310 installed or otherwise providedby the operating system of the client 102.

The network stack 310 comprises any type and form of interfaces forreceiving, obtaining, providing or otherwise accessing any informationand data related to network communications of the client 102. In oneembodiment, an interface to the network stack 310 comprises anapplication programming interface (API). The interface may also compriseany function call, hooking or filtering mechanism, event or call backmechanism, or any type of interfacing technique. The network stack 310via the interface may receive or provide any type and form of datastructure, such as an object, related to functionality or operation ofthe network stack 310. For example, the data structure may compriseinformation and data related to a network packet or one or more networkpackets. In some embodiments, the data structure comprises a portion ofthe network packet processed at a protocol layer of the network stack310, such as a network packet of the transport layer. In someembodiments, the data structure 325 comprises a kernel-level datastructure, while in other embodiments, the data structure 325 comprisesa user-mode data structure. A kernel-level data structure may comprise adata structure obtained or related to a portion of the network stack 310operating in kernel-mode 302, or a network driver or other softwarerunning in kernel-mode 302, or any data structure obtained or receivedby a service, process, task, thread or other executable instructionsrunning or operating in kernel-mode of the operating system.

Additionally, some portions of the network stack 310 may execute oroperate in kernel-mode 302, for example, the data link or network layer,while other portions execute or operate in user-mode 303, such as anapplication layer of the network stack 310. For example, a first portion310 a of the network stack may provide user-mode access to the networkstack 310 to an application while a second portion 310 a of the networkstack 310 provides access to a network. In some embodiments, a firstportion 310 a of the network stack may comprise one or more upper layersof the network stack 310, such as any of layers 5-7. In otherembodiments, a second portion 310 b of the network stack 310 comprisesone or more lower layers, such as any of layers 1-4. Each of the firstportion 310 a and second portion 310 b of the network stack 310 maycomprise any portion of the network stack 310, at any one or morenetwork layers, in user-mode 203, kernel-mode, 202, or combinationsthereof, or at any portion of a network layer or interface point to anetwork layer or any portion of or interface point to the user-mode 203and kernel-mode 203.

The interceptor 350 may comprise software, hardware, or any combinationof software and hardware. In one embodiment, the interceptor 350intercept a network communication at any point in the network stack 310,and redirects or transmits the network communication to a destinationdesired, managed or controlled by the interceptor 350 or client agent120. For example, the interceptor 350 may intercept a networkcommunication of a network stack 310 of a first network and transmit thenetwork communication to the appliance 200 for transmission on a secondnetwork 104. In some embodiments, the interceptor 350 comprises any typeinterceptor 350 comprises a driver, such as a network driver constructedand designed to interface and work with the network stack 310. In someembodiments, the client agent 120 and/or interceptor 350 operates at oneor more layers of the network stack 310, such as at the transport layer.In one embodiment, the interceptor 350 comprises a filter driver,hooking mechanism, or any form and type of suitable network driverinterface that interfaces to the transport layer of the network stack,such as via the transport driver interface (TDI). In some embodiments,the interceptor 350 interfaces to a first protocol layer, such as thetransport layer and another protocol layer, such as any layer above thetransport protocol layer, for example, an application protocol layer. Inone embodiment, the interceptor 350 may comprise a driver complying withthe Network Driver Interface Specification (NDIS), or a NDIS driver. Inanother embodiment, the interceptor 350 may comprise a min-filter or amini-port driver. In one embodiment, the interceptor 350, or portionthereof, operates in kernel-mode 202. In another embodiment, theinterceptor 350, or portion thereof, operates in user-mode 203. In someembodiments, a portion of the interceptor 350 operates in kernel-mode202 while another portion of the interceptor 350 operates in user-mode203. In other embodiments, the client agent 120 operates in user-mode203 but interfaces via the interceptor 350 to a kernel-mode driver,process, service, task or portion of the operating system, such as toobtain a kernel-level data structure 225. In further embodiments, theinterceptor 350 is a user-mode application or program, such asapplication.

In one embodiment, the interceptor 350 intercepts any transport layerconnection requests. In these embodiments, the interceptor 350 executetransport layer application programming interface (API) calls to set thedestination information, such as destination IP address and/or port to adesired location for the location. In this manner, the interceptor 350intercepts and redirects the transport layer connection to a IP addressand port controlled or managed by the interceptor 350 or client agent120. In one embodiment, the interceptor 350 sets the destinationinformation for the connection to a local IP address and port of theclient 102 on which the client agent 120 is listening. For example, theclient agent 120 may comprise a proxy service listening on a local IPaddress and port for redirected transport layer communications. In someembodiments, the client agent 120 then communicates the redirectedtransport layer communication to the appliance 200.

In some embodiments, the interceptor 350 intercepts a Domain NameService (DNS) request. In one embodiment, the client agent 120 and/orinterceptor 350 resolves the DNS request. In another embodiment, theinterceptor transmits the intercepted DNS request to the appliance 200for DNS resolution. In one embodiment, the appliance 200 resolves theDNS request and communicates the DNS response to the client agent 120.In some embodiments, the appliance 200 resolves the DNS request viaanother appliance 200′ or a DNS server 106.

In yet another embodiment, the client agent 120 may comprise two agents120 and 120′. In one embodiment, a first agent 120 may comprise aninterceptor 350 operating at the network layer of the network stack 310.In some embodiments, the first agent 120 intercepts network layerrequests such as Internet Control Message Protocol (ICMP) requests(e.g., ping and traceroute). In other embodiments, the second agent 120′may operate at the transport layer and intercept transport layercommunications. In some embodiments, the first agent 120 interceptscommunications at one layer of the network stack 210 and interfaces withor communicates the intercepted communication to the second agent 120′.

The client agent 120 and/or interceptor 350 may operate at or interfacewith a protocol layer in a manner transparent to any other protocollayer of the network stack 310. For example, in one embodiment, theinterceptor 350 operates or interfaces with the transport layer of thenetwork stack 310 transparently to any protocol layer below thetransport layer, such as the network layer, and any protocol layer abovethe transport layer, such as the session, presentation or applicationlayer protocols. This allows the other protocol layers of the networkstack 310 to operate as desired and without modification for using theinterceptor 350. As such, the client agent 120 and/or interceptor 350can interface with the transport layer to secure, optimize, accelerate,route or load-balance any communications provided via any protocolcarried by the transport layer, such as any application layer protocolover TCP/IP.

Furthermore, the client agent 120 and/or interceptor may operate at orinterface with the network stack 310 in a manner transparent to anyapplication, a user of the client 102, and any other computing device,such as a server, in communications with the client 102. The clientagent 120 and/or interceptor 350 may be installed and/or executed on theclient 102 in a manner without modification of an application. In someembodiments, the user of the client 102 or a computing device incommunications with the client 102 are not aware of the existence,execution or operation of the client agent 120 and/or interceptor 350.As such, in some embodiments, the client agent 120 and/or interceptor350 is installed, executed, and/or operated transparently to anapplication, user of the client 102, another computing device, such as aserver, or any of the protocol layers above and/or below the protocollayer interfaced to by the interceptor 350.

The client agent 120 includes an acceleration program 302, a streamingclient 306, a collection agent 304, and/or monitoring agent 197. In oneembodiment, the client agent 120 comprises an Independent ComputingArchitecture (ICA) client, or any portion thereof, developed by CitrixSystems, Inc. of Fort Lauderdale, Fla., and is also referred to as anICA client. In some embodiments, the client 120 comprises an applicationstreaming client 306 for streaming an application from a server 106 to aclient 102. In some embodiments, the client agent 120 comprises anacceleration program 302 for accelerating communications between client102 and server 106. In another embodiment, the client agent 120 includesa collection agent 304 for performing end-point detection/scanning andcollecting end-point information for the appliance 200 and/or server106.

In some embodiments, the acceleration program 302 comprises aclient-side acceleration program for performing one or more accelerationtechniques to accelerate, enhance or otherwise improve a client'scommunications with and/or access to a server 106, such as accessing anapplication provided by a server 106. The logic, functions, and/oroperations of the executable instructions of the acceleration program302 may perform one or more of the following acceleration techniques: 1)multi-protocol compression, 2) transport control protocol pooling, 3)transport control protocol multiplexing, 4) transport control protocolbuffering, and 5) caching via a cache manager. Additionally, theacceleration program 302 may perform encryption and/or decryption of anycommunications received and/or transmitted by the client 102. In someembodiments, the acceleration program 302 performs one or more of theacceleration techniques in an integrated manner or fashion.Additionally, the acceleration program 302 can perform compression onany of the protocols, or multiple-protocols, carried as a payload of anetwork packet of the transport layer protocol.

The streaming client 306 comprises an application, program, process,service, task or executable instructions for receiving and executing astreamed application from a server 106. A server 106 may stream one ormore application data files to the streaming client 306 for playing,executing or otherwise causing to be executed the application on theclient 102. In some embodiments, the server 106 transmits a set ofcompressed or packaged application data files to the streaming client306. In some embodiments, the plurality of application files arecompressed and stored on a file server within an archive file such as aCAB, ZIP, SIT, TAR, JAR or other archive. In one embodiment, the server106 decompresses, unpackages or unarchives the application files andtransmits the files to the client 102. In another embodiment, the client102 decompresses, unpackages or unarchives the application files. Thestreaming client 306 dynamically installs the application, or portionthereof, and executes the application. In one embodiment, the streamingclient 306 may be an executable program. In some embodiments, thestreaming client 306 may be able to launch another executable program.

The collection agent 304 comprises an application, program, process,service, task or executable instructions for identifying, obtainingand/or collecting information about the client 102. In some embodiments,the appliance 200 transmits the collection agent 304 to the client 102or client agent 120. The collection agent 304 may be configuredaccording to one or more policies of the policy engine 236 of theappliance. In other embodiments, the collection agent 304 transmitscollected information on the client 102 to the appliance 200. In oneembodiment, the policy engine 236 of the appliance 200 uses thecollected information to determine and provide access, authenticationand authorization control of the client's connection to a network 104.

In one embodiment, the collection agent 304 comprises an end-pointdetection and scanning mechanism, which identifies and determines one ormore attributes or characteristics of the client. For example, thecollection agent 304 may identify and determine any one or more of thefollowing client-side attributes: 1) the operating system an/or aversion of an operating system, 2) a service pack of the operatingsystem, 3) a running service, 4) a running process, and 5) a file. Thecollection agent 304 may also identify and determine the presence orversions of any one or more of the following on the client: 1) antivirussoftware, 2) personal firewall software, 3) anti-spam software, and 4)internet security software. The policy engine 236 may have one or morepolicies based on any one or more of the attributes or characteristicsof the client or client-side attributes.

In some embodiments, the client agent 120 includes a monitoring agent197 as discussed in conjunction with FIGS. 1D and 2B. The monitoringagent 197 may be any type and form of script, such as Visual Basic orJava script. In one embodiment, the monitoring agent 129 monitors andmeasures performance of any portion of the client agent 120. Forexample, in some embodiments, the monitoring agent 129 monitors andmeasures performance of the acceleration program 302. In anotherembodiment, the monitoring agent 129 monitors and measures performanceof the streaming client 306. In other embodiments, the monitoring agent129 monitors and measures performance of the collection agent 304. Instill another embodiment, the monitoring agent 129 monitors and measuresperformance of the interceptor 350. In some embodiments, the monitoringagent 129 monitors and measures any resource of the client 102, such asmemory, CPU and disk.

The monitoring agent 197 may monitor and measure performance of anyapplication of the client. In one embodiment, the monitoring agent 129monitors and measures performance of a browser on the client 102. Insome embodiments, the monitoring agent 197 monitors and measuresperformance of any application delivered via the client agent 120. Inother embodiments, the monitoring agent 197 measures and monitors enduser response times for an application, such as web-based or HTTPresponse times. The monitoring agent 197 may monitor and measureperformance of an ICA or RDP client. In another embodiment, themonitoring agent 197 measures and monitors metrics for a user session orapplication session. In some embodiments, monitoring agent 197 measuresand monitors an ICA or RDP session. In one embodiment, the monitoringagent 197 measures and monitors the performance of the appliance 200 inaccelerating delivery of an application and/or data to the client 102.

In some embodiments and still referring to FIG. 3, a first program 322may be used to install and/or execute the client agent 120, or portionthereof, such as the interceptor 350, automatically, silently,transparently, or otherwise. In one embodiment, the first program 322comprises a plugin component, such an ActiveX control or Java control orscript that is loaded into and executed by an application. For example,the first program comprises an ActiveX control loaded and run by a webbrowser application, such as in the memory space or context of theapplication. In another embodiment, the first program 322 comprises aset of executable instructions loaded into and run by the application,such as a browser. In one embodiment, the first program 322 comprises adesigned and constructed program to install the client agent 120. Insome embodiments, the first program 322 obtains, downloads, or receivesthe client agent 120 via the network from another computing device. Inanother embodiment, the first program 322 is an installer program or aplug and play manager for installing programs, such as network drivers,on the operating system of the client 102.

D. Systems and Methods for Providing Virtualized Application DeliveryController

Referring now to FIG. 4A, a block diagram depicts one embodiment of avirtualization environment 400. In brief overview, a computing device100 includes a hypervisor layer, a virtualization layer, and a hardwarelayer. The hypervisor layer includes a hypervisor 401 (also referred toas a virtualization manager) that allocates and manages access to anumber of physical resources in the hardware layer (e.g., theprocessor(s) 421, and disk(s) 428) by at least one virtual machineexecuting in the virtualization layer. The virtualization layer includesat least one operating system 410 and a plurality of virtual resourcesallocated to the at least one operating system 410. Virtual resourcesmay include, without limitation, a plurality of virtual processors 432a, 432 b, 432 c (generally 432), and virtual disks 442 a, 442 b, 442 c(generally 442), as well as virtual resources such as virtual memory andvirtual network interfaces. The plurality of virtual resources and theoperating system 410 may be referred to as a virtual machine 406. Avirtual machine 406 may include a control operating system 405 incommunication with the hypervisor 401 and used to execute applicationsfor managing and configuring other virtual machines on the computingdevice 100.

In greater detail, a hypervisor 401 may provide virtual resources to anoperating system in any manner which simulates the operating systemhaving access to a physical device. A hypervisor 401 may provide virtualresources to any number of guest operating systems 410 a, 410 b(generally 410). In some embodiments, a computing device 100 executesone or more types of hypervisors. In these embodiments, hypervisors maybe used to emulate virtual hardware, partition physical hardware,virtualize physical hardware, and execute virtual machines that provideaccess to computing environments. Hypervisors may include thosemanufactured by VMWare, Inc., of Palo Alto, Calif.; the XEN hypervisor,an open source product whose development is overseen by the open sourceXen.org community; HyperV, VirtualServer or virtual PC hypervisorsprovided by Microsoft, or others. In some embodiments, a computingdevice 100 executing a hypervisor that creates a virtual machineplatform on which guest operating systems may execute is referred to asa host server. In one of these embodiments, for example, the computingdevice 100 is a XEN SERVER provided by Citrix Systems, Inc., of FortLauderdale, Fla.

In some embodiments, a hypervisor 401 executes within an operatingsystem executing on a computing device. In one of these embodiments, acomputing device executing an operating system and a hypervisor 401 maybe said to have a host operating system (the operating system executingon the computing device), and a guest operating system (an operatingsystem executing within a computing resource partition provided by thehypervisor 401). In other embodiments, a hypervisor 401 interactsdirectly with hardware on a computing device, instead of executing on ahost operating system. In one of these embodiments, the hypervisor 401may be said to be executing on “bare metal,” referring to the hardwarecomprising the computing device.

In some embodiments, a hypervisor 401 may create a virtual machine 406a-c (generally 406) in which an operating system 410 executes. In one ofthese embodiments, for example, the hypervisor 401 loads a virtualmachine image to create a virtual machine 406. In another of theseembodiments, the hypervisor 401 executes an operating system 410 withinthe virtual machine 406. In still another of these embodiments, thevirtual machine 406 executes an operating system 410.

In some embodiments, the hypervisor 401 controls processor schedulingand memory partitioning for a virtual machine 406 executing on thecomputing device 100. In one of these embodiments, the hypervisor 401controls the execution of at least one virtual machine 406. In anotherof these embodiments, the hypervisor 401 presents at least one virtualmachine 406 with an abstraction of at least one hardware resourceprovided by the computing device 100. In other embodiments, thehypervisor 401 controls whether and how physical processor capabilitiesare presented to the virtual machine 406.

A control operating system 405 may execute at least one application formanaging and configuring the guest operating systems. In one embodiment,the control operating system 405 may execute an administrativeapplication, such as an application including a user interface providingadministrators with access to functionality for managing the executionof a virtual machine, including functionality for executing a virtualmachine, terminating an execution of a virtual machine, or identifying atype of physical resource for allocation to the virtual machine. Inanother embodiment, the hypervisor 401 executes the control operatingsystem 405 within a virtual machine 406 created by the hypervisor 401.In still another embodiment, the control operating system 405 executesin a virtual machine 406 that is authorized to directly access physicalresources on the computing device 100. In some embodiments, a controloperating system 405 a on a computing device 100 a may exchange datawith a control operating system 405 b on a computing device 100 b, viacommunications between a hypervisor 401 a and a hypervisor 401 b. Inthis way, one or more computing devices 100 may exchange data with oneor more of the other computing devices 100 regarding processors andother physical resources available in a pool of resources. In one ofthese embodiments, this functionality allows a hypervisor to manage apool of resources distributed across a plurality of physical computingdevices. In another of these embodiments, multiple hypervisors manageone or more of the guest operating systems executed on one of thecomputing devices 100.

In one embodiment, the control operating system 405 executes in avirtual machine 406 that is authorized to interact with at least oneguest operating system 410. In another embodiment, a guest operatingsystem 410 communicates with the control operating system 405 via thehypervisor 401 in order to request access to a disk or a network. Instill another embodiment, the guest operating system 410 and the controloperating system 405 may communicate via a communication channelestablished by the hypervisor 401, such as, for example, via a pluralityof shared memory pages made available by the hypervisor 401.

In some embodiments, the control operating system 405 includes a networkback-end driver for communicating directly with networking hardwareprovided by the computing device 100. In one of these embodiments, thenetwork back-end driver processes at least one virtual machine requestfrom at least one guest operating system 110. In other embodiments, thecontrol operating system 405 includes a block back-end driver forcommunicating with a storage element on the computing device 100. In oneof these embodiments, the block back-end driver reads and writes datafrom the storage element based upon at least one request received from aguest operating system 410.

In one embodiment, the control operating system 405 includes a toolsstack 404. In another embodiment, a tools stack 404 providesfunctionality for interacting with the hypervisor 401, communicatingwith other control operating systems 405 (for example, on a secondcomputing device 100), or managing virtual machines 406 b, 406 c on thecomputing device 100. In another embodiment, the tools stack 404includes customized applications for providing improved managementfunctionality to an administrator of a virtual machine farm. In someembodiments, at least one of the tools stack 404 and the controloperating system 405 include a management API that provides an interfacefor remotely configuring and controlling virtual machines 406 running ona computing device 100. In other embodiments, the control operatingsystem 405 communicates with the hypervisor 401 through the tools stack104.

In one embodiment, the hypervisor 401 executes a guest operating system410 within a virtual machine 406 created by the hypervisor 401. Inanother embodiment, the guest operating system 410 provides a user ofthe computing device 100 with access to resources within a computingenvironment. In still another embodiment, a resource includes a program,an application, a document, a file, a plurality of applications, aplurality of files, an executable program file, a desktop environment, acomputing environment, or other resource made available to a user of thecomputing device 100. In yet another embodiment, the resource may bedelivered to the computing device 100 via a plurality of access methodsincluding, but not limited to, conventional installation directly on thecomputing device 100, delivery to the computing device 100 via a methodfor application streaming, delivery to the computing device 100 ofoutput data generated by an execution of the resource on a secondcomputing device 100′ and communicated to the computing device 100 via apresentation layer protocol, delivery to the computing device 100 ofoutput data generated by an execution of the resource via a virtualmachine executing on a second computing device 100′, or execution from aremovable storage device connected to the computing device 100, such asa USB device, or via a virtual machine executing on the computing device100 and generating output data. In some embodiments, the computingdevice 100 transmits output data generated by the execution of theresource to another computing device 100′.

In one embodiment, the guest operating system 410, in conjunction withthe virtual machine on which it executes, forms a fully-virtualizedvirtual machine which is not aware that it is a virtual machine; such amachine may be referred to as a “Domain U HVM (Hardware Virtual Machine)virtual machine”. In another embodiment, a fully-virtualized machineincludes software emulating a Basic Input/Output System (BIOS) in orderto execute an operating system within the fully-virtualized machine. Instill another embodiment, a fully-virtualized machine may include adriver that provides functionality by communicating with the hypervisor401. In such an embodiment, the driver may be aware that it executeswithin a virtualized environment. In another embodiment, the guestoperating system 410, in conjunction with the virtual machine on whichit executes, forms a paravirtualized virtual machine, which is awarethat it is a virtual machine; such a machine may be referred to as a“Domain U PV virtual machine”. In another embodiment, a paravirtualizedmachine includes additional drivers that a fully-virtualized machinedoes not include. In still another embodiment, the paravirtualizedmachine includes the network back-end driver and the block back-enddriver included in a control operating system 405, as described above.

Referring now to FIG. 4B, a block diagram depicts one embodiment of aplurality of networked computing devices in a system in which at leastone physical host executes a virtual machine. In brief overview, thesystem includes a management component 404 and a hypervisor 401. Thesystem includes a plurality of computing devices 100, a plurality ofvirtual machines 406, a plurality of hypervisors 401, a plurality ofmanagement components referred to as tools stacks 404, and a physicalresource 421, 428. The plurality of physical machines 100 may each beprovided as computing devices 100, described above in connection withFIGS. 1E-1H and 4A.

In greater detail, a physical disk 428 is provided by a computing device100 and stores at least a portion of a virtual disk 442. In someembodiments, a virtual disk 442 is associated with a plurality ofphysical disks 428. In one of these embodiments, one or more computingdevices 100 may exchange data with one or more of the other computingdevices 100 regarding processors and other physical resources availablein a pool of resources, allowing a hypervisor to manage a pool ofresources distributed across a plurality of physical computing devices.In some embodiments, a computing device 100 on which a virtual machine406 executes is referred to as a physical host 100 or as a host machine100.

The hypervisor executes on a processor on the computing device 100. Thehypervisor allocates, to a virtual disk, an amount of access to thephysical disk. In one embodiment, the hypervisor 401 allocates an amountof space on the physical disk. In another embodiment, the hypervisor 401allocates a plurality of pages on the physical disk. In someembodiments, the hypervisor provisions the virtual disk 442 as part of aprocess of initializing and executing a virtual machine 450.

In one embodiment, the management component 404 a is referred to as apool management component 404 a. In another embodiment, a managementoperating system 405 a, which may be referred to as a control operatingsystem 405 a, includes the management component. In some embodiments,the management component is referred to as a tools stack. In one ofthese embodiments, the management component is the tools stack 404described above in connection with FIG. 4A. In other embodiments, themanagement component 404 provides a user interface for receiving, from auser such as an administrator, an identification of a virtual machine406 to provision and/or execute. In still other embodiments, themanagement component 404 provides a user interface for receiving, from auser such as an administrator, the request for migration of a virtualmachine 406 b from one physical machine 100 to another. In furtherembodiments, the management component 404 a identifies a computingdevice 100 b on which to execute a requested virtual machine 406 d andinstructs the hypervisor 401 b on the identified computing device 100 bto execute the identified virtual machine; such a management componentmay be referred to as a pool management component.

Referring now to FIG. 4C, embodiments of a virtual application deliverycontroller or virtual appliance 450 are depicted. In brief overview, anyof the functionality and/or embodiments of the appliance 200 (e.g., anapplication delivery controller) described above in connection withFIGS. 2A and 2B may be deployed in any embodiment of the virtualizedenvironment described above in connection with FIGS. 4A and 4B. Insteadof the functionality of the application delivery controller beingdeployed in the form of an appliance 200, such functionality may bedeployed in a virtualized environment 400 on any computing device 100,such as a client 102, server 106 or appliance 200.

Referring now to FIG. 4C, a diagram of an embodiment of a virtualappliance 450 operating on a hypervisor 401 of a server 106 is depicted.As with the appliance 200 of FIGS. 2A and 2B, the virtual appliance 450may provide functionality for availability, performance, offload andsecurity. For availability, the virtual appliance may perform loadbalancing between layers 4 and 7 of the network and may also performintelligent service health monitoring. For performance increases vianetwork traffic acceleration, the virtual appliance may perform cachingand compression. To offload processing of any servers, the virtualappliance may perform connection multiplexing and pooling and/or SSLprocessing. For security, the virtual appliance may perform any of theapplication firewall functionality and SSL VPN function of appliance200.

Any of the modules of the appliance 200 as described in connection withFIG. 2A may be packaged, combined, designed or constructed in a form ofthe virtualized appliance delivery controller 450 deployable as one ormore software modules or components executable in a virtualizedenvironment 300 or non-virtualized environment on any server, such as anoff the shelf server. For example, the virtual appliance may be providedin the form of an installation package to install on a computing device.With reference to FIG. 2A, any of the cache manager 232, policy engine236, compression 238, encryption engine 234, packet engine 240, GUI 210,CLI 212, shell services 214 and health monitoring programs 216 may bedesigned and constructed as a software component or module to run on anyoperating system of a computing device and/or of a virtualizedenvironment 300. Instead of using the encryption processor 260,processor 262, memory 264 and network stack 267 of the appliance 200,the virtualized appliance 400 may use any of these resources as providedby the virtualized environment 400 or as otherwise available on theserver 106.

Still referring to FIG. 4C, and in brief overview, any one or morevServers 275A-275N may be in operation or executed in a virtualizedenvironment 400 of any type of computing device 100, such as any server106. Any of the modules or functionality of the appliance 200 describedin connection with FIG. 2B may be designed and constructed to operate ineither a virtualized or non-virtualized environment of a server. Any ofthe vServer 275, SSL VPN 280, Intranet UP 282, Switching 284, DNS 286,acceleration 288, App FW 280 and monitoring agent may be packaged,combined, designed or constructed in a form of application deliverycontroller 450 deployable as one or more software modules or componentsexecutable on a device and/or virtualized environment 400.

In some embodiments, a server may execute multiple virtual machines 406a-406 n in the virtualization environment with each virtual machinerunning the same or different embodiments of the virtual applicationdelivery controller 450. In some embodiments, the server may execute oneor more virtual appliances 450 on one or more virtual machines on a coreof a multi-core processing system. In some embodiments, the server mayexecute one or more virtual appliances 450 on one or more virtualmachines on each processor of a multiple processor device.

E. Systems and Methods for Providing A Multi-Core Architecture

In accordance with Moore's Law, the number of transistors that may beplaced on an integrated circuit may double approximately every twoyears. However, CPU speed increases may reach plateaus, for example CPUspeed has been around 3.5-4 GHz range since 2005. In some cases, CPUmanufacturers may not rely on CPU speed increases to gain additionalperformance. Some CPU manufacturers may add additional cores to theirprocessors to provide additional performance. Products, such as those ofsoftware and networking vendors, that rely on CPUs for performance gainsmay improve their performance by leveraging these multi-core CPUs. Thesoftware designed and constructed for a single CPU may be redesignedand/or rewritten to take advantage of a multi-threaded, parallelarchitecture or otherwise a multi-core architecture.

A multi-core architecture of the appliance 200, referred to as nCore ormulti-core technology, allows the appliance in some embodiments to breakthe single core performance barrier and to leverage the power ofmulti-core CPUs. In the previous architecture described in connectionwith FIG. 2A, a single network or packet engine is run. The multiplecores of the nCore technology and architecture allow multiple packetengines to run concurrently and/or in parallel. With a packet enginerunning on each core, the appliance architecture leverages theprocessing capacity of additional cores. In some embodiments, thisprovides up to a 7× increase in performance and scalability.

Illustrated in FIG. 5A are some embodiments of work, task, load ornetwork traffic distribution across one or more processor coresaccording to a type of parallelism or parallel computing scheme, such asfunctional parallelism, data parallelism or flow-based data parallelism.In brief overview, FIG. 5A illustrates embodiments of a multi-coresystem such as an appliance 200′ with n-cores, a total of cores numbers1 through N. In one embodiment, work, load or network traffic can bedistributed among a first core 505A, a second core 505B, a third core505C, a fourth core 505D, a fifth core 505E, a sixth core 505F, aseventh core 505G, and so on such that distribution is across all or twoor more of the n cores 505N (hereinafter referred to collectively ascores 505.) There may be multiple VIPs 275 each running on a respectivecore of the plurality of cores. There may be multiple packet engines 240each running on a respective core of the plurality of cores. Any of theapproaches used may lead to different, varying or similar work load orperformance level 515 across any of the cores. For a functionalparallelism approach, each core may run a different function of thefunctionalities provided by the packet engine, a VIP 275 or appliance200. In a data parallelism approach, data may be paralleled ordistributed across the cores based on the Network Interface Card (NIC)or VIP 275 receiving the data. In another data parallelism approach,processing may be distributed across the cores by distributing dataflows to each core.

In further detail to FIG. 5A, in some embodiments, load, work or networktraffic can be distributed among cores 505 according to functionalparallelism 500. Functional parallelism may be based on each coreperforming one or more respective functions. In some embodiments, afirst core may perform a first function while a second core performs asecond function. In functional parallelism approach, the functions to beperformed by the multi-core system are divided and distributed to eachcore according to functionality. In some embodiments, functionalparallelism may be referred to as task parallelism and may be achievedwhen each processor or core executes a different process or function onthe same or different data. The core or processor may execute the sameor different code. In some cases, different execution threads or codemay communicate with one another as they work. Communication may takeplace to pass data from one thread to the next as part of a workflow.

In some embodiments, distributing work across the cores 505 according tofunctional parallelism 500, can comprise distributing network trafficaccording to a particular function such as network input/outputmanagement (NW I/O) 510A, secure sockets layer (SSL) encryption anddecryption 510B and transmission control protocol (TCP) functions 510C.This may lead to a work, performance or computing load 515 based on avolume or level of functionality being used. In some embodiments,distributing work across the cores 505 according to data parallelism540, can comprise distributing an amount of work 515 based ondistributing data associated with a particular hardware or softwarecomponent. In some embodiments, distributing work across the cores 505according to flow-based data parallelism 520, can comprise distributingdata based on a context or flow such that the amount of work 515A-N oneach core may be similar, substantially equal or relatively evenlydistributed.

In the case of the functional parallelism approach, each core may beconfigured to run one or more functionalities of the plurality offunctionalities provided by the packet engine or VIP of the appliance.For example, core 1 may perform network I/O processing for the appliance200′ while core 2 performs TCP connection management for the appliance.Likewise, core 3 may perform SSL offloading while core 4 may performlayer 7 or application layer processing and traffic management. Each ofthe cores may perform the same function or different functions. Each ofthe cores may perform more than one function. Any of the cores may runany of the functionality or portions thereof identified and/or describedin conjunction with FIGS. 2A and 2B. In this the approach, the workacross the cores may be divided by function in either a coarse-grainedor fine-grained manner. In some cases, as illustrated in FIG. 5A,division by function may lead to different cores running at differentlevels of performance or load 515.

In the case of the functional parallelism approach, each core may beconfigured to run one or more functionalities of the plurality offunctionalities provided by the packet engine of the appliance. Forexample, core 1 may perform network I/O processing for the appliance200′ while core 2 performs TCP connection management for the appliance.Likewise, core 3 may perform SSL offloading while core 4 may performlayer 7 or application layer processing and traffic management. Each ofthe cores may perform the same function or different functions. Each ofthe cores may perform more than one function. Any of the cores may runany of the functionality or portions thereof identified and/or describedin conjunction with FIGS. 2A and 2B. In this the approach, the workacross the cores may be divided by function in either a coarse-grainedor fine-grained manner. In some cases, as illustrated in FIG. 5Adivision by function may lead to different cores running at differentlevels of load or performance.

The functionality or tasks may be distributed in any arrangement andscheme. For example, FIG. 5B illustrates a first core, Core 1 505A,processing applications and processes associated with network I/Ofunctionality 510A. Network traffic associated with network I/O, in someembodiments, can be associated with a particular port number. Thus,outgoing and incoming packets having a port destination associated withNW I/O 510A will be directed towards Core 1 505A which is dedicated tohandling all network traffic associated with the NW I/O port. Similarly,Core 2 505B is dedicated to handling functionality associated with SSLprocessing and Core 4 505D may be dedicated handling all TCP levelprocessing and functionality.

While FIG. 5A illustrates functions such as network I/O, SSL and TCP,other functions can be assigned to cores. These other functions caninclude any one or more of the functions or operations described herein.For example, any of the functions described in conjunction with FIGS. 2Aand 2B may be distributed across the cores on a functionality basis. Insome cases, a first VIP 275A may run on a first core while a second VIP275B with a different configuration may run on a second core. In someembodiments, each core 505 can handle a particular functionality suchthat each core 505 can handle the processing associated with thatparticular function. For example, Core 2 505B may handle SSL offloadingwhile Core 4 505D may handle application layer processing and trafficmanagement.

In other embodiments, work, load or network traffic may be distributedamong cores 505 according to any type and form of data parallelism 540.In some embodiments, data parallelism may be achieved in a multi-coresystem by each core performing the same task or functionally ondifferent pieces of distributed data. In some embodiments, a singleexecution thread or code controls operations on all pieces of data. Inother embodiments, different threads or instructions control theoperation, but may execute the same code. In some embodiments, dataparallelism is achieved from the perspective of a packet engine,vServers (VIPs) 275A-C, network interface cards (NIC) 542D-E and/or anyother networking hardware or software included on or associated with anappliance 200. For example, each core may run the same packet engine orVIP code or configuration but operate on different sets of distributeddata. Each networking hardware or software construct can receivedifferent, varying or substantially the same amount of data, and as aresult may have varying, different or relatively the same amount of load515

In the case of a data parallelism approach, the work may be divided upand distributed based on VIPs, NICs and/or data flows of the VIPs orNICs. In one of these approaches, the work of the multi-core system maybe divided or distributed among the VIPs by having each VIP work on adistributed set of data. For example, each core may be configured to runone or more VIPs. Network traffic may be distributed to the core foreach VIP handling that traffic. In another of these approaches, the workof the appliance may be divided or distributed among the cores based onwhich NIC receives the network traffic. For example, network traffic ofa first NIC may be distributed to a first core while network traffic ofa second NIC may be distributed to a second core. In some cases, a coremay process data from multiple NICs.

While FIG. 5A illustrates a single vServer associated with a single core505, as is the case for VIP1 275A, VIP2 275B and VIP3 275C. In someembodiments, a single vServer can be associated with one or more cores505. In contrast, one or more vServers can be associated with a singlecore 505. Associating a vServer with a core 505 may include that core505 to process all functions associated with that particular vServer. Insome embodiments, each core executes a VIP having the same code andconfiguration. In other embodiments, each core executes a VIP having thesame code but different configuration. In some embodiments, each coreexecutes a VIP having different code and the same or differentconfiguration.

Like vServers, NICs can also be associated with particular cores 505. Inmany embodiments, NICs can be connected to one or more cores 505 suchthat when a NIC receives or transmits data packets, a particular core505 handles the processing involved with receiving and transmitting thedata packets. In one embodiment, a single NIC can be associated with asingle core 505, as is the case with NIC1 542D and NIC2 542E. In otherembodiments, one or more NICs can be associated with a single core 505.In other embodiments, a single NIC can be associated with one or morecores 505. In these embodiments, load could be distributed amongst theone or more cores 505 such that each core 505 processes a substantiallysimilar amount of load. A core 505 associated with a NIC may process allfunctions and/or data associated with that particular NIC.

While distributing work across cores based on data of VIPs or NICs mayhave a level of independency, in some embodiments, this may lead tounbalanced use of cores as illustrated by the varying loads 515 of FIG.5A.

In some embodiments, load, work or network traffic can be distributedamong cores 505 based on any type and form of data flow. In another ofthese approaches, the work may be divided or distributed among coresbased on data flows. For example, network traffic between a client and aserver traversing the appliance may be distributed to and processed byone core of the plurality of cores. In some cases, the core initiallyestablishing the session or connection may be the core for which networktraffic for that session or connection is distributed. In someembodiments, the data flow is based on any unit or portion of networktraffic, such as a transaction, a request/response communication ortraffic originating from an application on a client. In this manner andin some embodiments, data flows between clients and servers traversingthe appliance 200′ may be distributed in a more balanced manner than theother approaches.

In flow-based data parallelism 520, distribution of data is related toany type of flow of data, such as request/response pairings,transactions, sessions, connections or application communications. Forexample, network traffic between a client and a server traversing theappliance may be distributed to and processed by one core of theplurality of cores. In some cases, the core initially establishing thesession or connection may be the core for which network traffic for thatsession or connection is distributed. The distribution of data flow maybe such that each core 505 carries a substantially equal or relativelyevenly distributed amount of load, data or network traffic.

In some embodiments, the data flow is based on any unit or portion ofnetwork traffic, such as a transaction, a request/response communicationor traffic originating from an application on a client. In this mannerand in some embodiments, data flows between clients and serverstraversing the appliance 200′ may be distributed in a more balancedmanner than the other approached. In one embodiment, data flow can bedistributed based on a transaction or a series of transactions. Thistransaction, in some embodiments, can be between a client and a serverand can be characterized by an IP address or other packet identifier.For example, Core 1 505A can be dedicated to transactions between aparticular client and a particular server, therefore the load 536A onCore 1 505A may be comprised of the network traffic associated with thetransactions between the particular client and server. Allocating thenetwork traffic to Core 1 505A can be accomplished by routing all datapackets originating from either the particular client or server to Core1 505A. While work or load can be distributed to the cores based in parton transactions, in other embodiments load or work can be allocated on aper packet basis. In these embodiments, the appliance 200 can interceptdata packets and allocate them to a core 505 having the least amount ofload. For example, the appliance 200 could allocate a first incomingdata packet to Core 1 505A because the load 536A on Core 1 is less thanthe load 536B-N on the rest of the cores 505B-N. Once the first datapacket is allocated to Core 1 505A, the amount of load 536A on Core 1505A is increased proportional to the amount of processing resourcesneeded to process the first data packet. When the appliance 200intercepts a second data packet, the appliance 200 will allocate theload to Core 4 505D because Core 4 505D has the second least amount ofload. Allocating data packets to the core with the least amount of loadcan, in some embodiments, ensure that the load 536A-N distributed toeach core 505 remains substantially equal.

In other embodiments, load can be allocated on a per unit basis where asection of network traffic is allocated to a particular core 505. Theabove-mentioned example illustrates load balancing on a per/packetbasis. In other embodiments, load can be allocated based on a number ofpackets such that every 10, 100 or 1000 packets are allocated to thecore 505 having the least amount of load. The number of packetsallocated to a core 505 can be a number determined by an application,user or administrator and can be any number greater than zero. In stillother embodiments, load can be allocated based on a time metric suchthat packets are distributed to a particular core 505 for apredetermined amount of time. In these embodiments, packets can bedistributed to a particular core 505 for five milliseconds or for anyperiod of time determined by a user, program, system, administrator orotherwise. After the predetermined time period elapses, data packets aretransmitted to a different core 505 for the predetermined period oftime.

Flow-based data parallelism methods for distributing work, load ornetwork traffic among the one or more cores 505 can comprise anycombination of the above-mentioned embodiments. These methods can becarried out by any part of the appliance 200, by an application or setof executable instructions executing on one of the cores 505, such asthe packet engine, or by any application, program or agent executing ona computing device in communication with the appliance 200.

The functional and data parallelism computing schemes illustrated inFIG. 5A can be combined in any manner to generate a hybrid parallelismor distributed processing scheme that encompasses function parallelism500, data parallelism 540, flow-based data parallelism 520 or anyportions thereof. In some cases, the multi-core system may use any typeand form of load balancing schemes to distribute load among the one ormore cores 505. The load balancing scheme may be used in any combinationwith any of the functional and data parallelism schemes or combinationsthereof.

Illustrated in FIG. 5B is an embodiment of a multi-core system 545,which may be any type and form of one or more systems, appliances,devices or components. This system 545, in some embodiments, can beincluded within an appliance 200 having one or more processing cores505A-N. The system 545 can further include one or more packet engines(PE) or packet processing engines (PPE) 548A-N communicating with amemory bus 556. The memory bus may be used to communicate with the oneor more processing cores 505A-N. Also included within the system 545 canbe one or more network interface cards (NIC) 552 and a flow distributor550 which can further communicate with the one or more processing cores505A-N. The flow distributor 550 can comprise a Receive Side Scaler(RSS) or Receive Side Scaling (RSS) module 560.

Further referring to FIG. 5B, and in more detail, in one embodiment thepacket engine(s) 548A-N can comprise any portion of the appliance 200described herein, such as any portion of the appliance described inFIGS. 2A and 2B. The packet engine(s) 548A-N can, in some embodiments,comprise any of the following elements: the packet engine 240, a networkstack 267; a cache manager 232; a policy engine 236; a compressionengine 238; an encryption engine 234; a GUI 210; a CLI 212; shellservices 214; monitoring programs 216; and any other software orhardware element able to receive data packets from one of either thememory bus 556 or the one of more cores 505A-N. In some embodiments, thepacket engine(s) 548A-N can comprise one or more vServers 275A-N, or anyportion thereof. In other embodiments, the packet engine(s) 548A-N canprovide any combination of the following functionalities: SSL VPN 280;Intranet UP 282; switching 284; DNS 286; packet acceleration 288; App FW280; monitoring such as the monitoring provided by a monitoring agent197; functionalities associated with functioning as a TCP stack; loadbalancing; SSL offloading and processing; content switching; policyevaluation; caching; compression; encoding; decompression; decoding;application firewall functionalities; XML processing and acceleration;and SSL VPN connectivity.

The packet engine(s) 548A-N can, in some embodiments, be associated witha particular server, user, client or network. When a packet engine 548becomes associated with a particular entity, that packet engine 548 canprocess data packets associated with that entity. For example, should apacket engine 548 be associated with a first user, that packet engine548 will process and operate on packets generated by the first user, orpackets having a destination address associated with the first user.Similarly, the packet engine 548 may choose not to be associated with aparticular entity such that the packet engine 548 can process andotherwise operate on any data packets not generated by that entity ordestined for that entity.

In some instances, the packet engine(s) 548A-N can be configured tocarry out the any of the functional and/or data parallelism schemesillustrated in FIG. 5A. In these instances, the packet engine(s) 548A-Ncan distribute functions or data among the processing cores 505A-N sothat the distribution is according to the parallelism or distributionscheme. In some embodiments, a single packet engine(s) 548A-N carriesout a load balancing scheme, while in other embodiments one or morepacket engine(s) 548A-N carry out a load balancing scheme. Each core505A-N, in one embodiment, can be associated with a particular packetengine 505 such that load balancing can be carried out by the packetengine 505. Load balancing may in this embodiment, require that eachpacket engine 505 associated with a core 505 communicate with the otherpacket engines 505 associated with cores 505 so that the packet engines505 can collectively determine where to distribute load. One embodimentof this process can include an arbiter that receives votes from eachpacket engine 505 for load. The arbiter can distribute load to eachpacket engine 505 based in part on the age of the engine's vote and insome cases a priority value associated with the current amount of loadon an engine's associated core 505.

Any of the packet engines running on the cores may run in user mode,kernel or any combination thereof. In some embodiments, the packetengine operates as an application or program running is user orapplication space. In these embodiments, the packet engine may use anytype and form of interface to access any functionality provided by thekernel. In some embodiments, the packet engine operates in kernel modeor as part of the kernel. In some embodiments, a first portion of thepacket engine operates in user mode while a second portion of the packetengine operates in kernel mode. In some embodiments, a first packetengine on a first core executes in kernel mode while a second packetengine on a second core executes in user mode. In some embodiments, thepacket engine or any portions thereof operates on or in conjunction withthe NIC or any drivers thereof.

In some embodiments the memory bus 556 can be any type and form ofmemory or computer bus. While a single memory bus 556 is depicted inFIG. 5B, the system 545 can comprise any number of memory buses 556. Inone embodiment, each packet engine 548 can be associated with one ormore individual memory buses 556.

The NIC 552 can in some embodiments be any of the network interfacecards or mechanisms described herein. The NIC 552 can have any number ofports. The NIC can be designed and constructed to connect to any typeand form of network 104. While a single NIC 552 is illustrated, thesystem 545 can comprise any number of NICs 552. In some embodiments,each core 505A-N can be associated with one or more single NICs 552.Thus, each core 505 can be associated with a single NIC 552 dedicated toa particular core 505. The cores 505A-N can comprise any of theprocessors described herein. Further, the cores 505A-N can be configuredaccording to any of the core 505 configurations described herein. Stillfurther, the cores 505A-N can have any of the core 505 functionalitiesdescribed herein. While FIG. 5B illustrates seven cores 505A-G, anynumber of cores 505 can be included within the system 545. Inparticular, the system 545 can comprise “N” cores, where “N” is a wholenumber greater than zero.

A core may have or use memory that is allocated or assigned for use tothat core. The memory may be considered private or local memory of thatcore and only accessible by that core. A core may have or use memorythat is shared or assigned to multiple cores. The memory may beconsidered public or shared memory that is accessible by more than onecore. A core may use any combination of private and public memory. Withseparate address spaces for each core, some level of coordination iseliminated from the case of using the same address space. With aseparate address space, a core can perform work on information and datain the core's own address space without worrying about conflicts withother cores. Each packet engine may have a separate memory pool for TCPand/or SSL connections.

Further referring to FIG. 5B, any of the functionality and/orembodiments of the cores 505 described above in connection with FIG. 5Acan be deployed in any embodiment of the virtualized environmentdescribed above in connection with FIGS. 4A and 4B. Instead of thefunctionality of the cores 505 being deployed in the form of a physicalprocessor 505, such functionality may be deployed in a virtualizedenvironment 400 on any computing device 100, such as a client 102,server 106 or appliance 200. In other embodiments, instead of thefunctionality of the cores 505 being deployed in the form of anappliance or a single device, the functionality may be deployed acrossmultiple devices in any arrangement. For example, one device maycomprise two or more cores and another device may comprise two or morecores. For example, a multi-core system may include a cluster ofcomputing devices, a server farm or network of computing devices. Insome embodiments, instead of the functionality of the cores 505 beingdeployed in the form of cores, the functionality may be deployed on aplurality of processors, such as a plurality of single core processors.

In one embodiment, the cores 505 may be any type and form of processor.In some embodiments, a core can function substantially similar to anyprocessor or central processing unit described herein. In someembodiment, the cores 505 may comprise any portion of any processordescribed herein. While FIG. 5A illustrates seven cores, there can existany “N” number of cores within an appliance 200, where “N” is any wholenumber greater than one. In some embodiments, the cores 505 can beinstalled within a common appliance 200, while in other embodiments thecores 505 can be installed within one or more appliance(s) 200communicatively connected to one another. The cores 505 can in someembodiments comprise graphics processing software, while in otherembodiments the cores 505 provide general processing capabilities. Thecores 505 can be installed physically near each other and/or can becommunicatively connected to each other. The cores may be connected byany type and form of bus or subsystem physically and/or communicativelycoupled to the cores for transferring data between to, from and/orbetween the cores.

While each core 505 can comprise software for communicating with othercores, in some embodiments a core manager (Not Shown) can facilitatecommunication between each core 505. In some embodiments, the kernel mayprovide core management. The cores may interface or communicate witheach other using a variety of interface mechanisms. In some embodiments,core to core messaging may be used to communicate between cores, such asa first core sending a message or data to a second core via a bus orsubsystem connecting the cores. In some embodiments, cores maycommunicate via any type and form of shared memory interface. In oneembodiment, there may be one or more memory locations shared among allthe cores. In some embodiments, each core may have separate memorylocations shared with each other core. For example, a first core mayhave a first shared memory with a second core and a second share memorywith a third core. In some embodiments, cores may communicate via anytype of programming or API, such as function calls via the kernel. Insome embodiments, the operating system may recognize and supportmultiple core devices and provide interfaces and API for inter-corecommunications.

The flow distributor 550 can be any application, program, library,script, task, service, process or any type and form of executableinstructions executing on any type and form of hardware. In someembodiments, the flow distributor 550 may any design and construction ofcircuitry to perform any of the operations and functions describedherein. In some embodiments, the flow distributor distribute, forwards,routes, controls and/ors manage the distribution of data packets amongthe cores 505 and/or packet engine or VIPs running on the cores. Theflow distributor 550, in some embodiments, can be referred to as aninterface master. In one embodiment, the flow distributor 550 comprisesa set of executable instructions executing on a core or processor of theappliance 200. In another embodiment, the flow distributor 550 comprisesa set of executable instructions executing on a computing machine incommunication with the appliance 200. In some embodiments, the flowdistributor 550 comprises a set of executable instructions executing ona NIC, such as firmware. In still other embodiments, the flowdistributor 550 comprises any combination of software and hardware todistribute data packets among cores or processors. In one embodiment,the flow distributor 550 executes on at least one of the cores 505A-N,while in other embodiments a separate flow distributor 550 assigned toeach core 505A-N executes on an associated core 505A-N. The flowdistributor may use any type and form of statistical or probabilisticalgorithms or decision making to balance the flows across the cores. Thehardware of the appliance, such as a NIC, or the kernel may be designedand constructed to support sequential operations across the NICs and/orcores.

In embodiments where the system 545 comprises one or more flowdistributors 550, each flow distributor 550 can be associated with aprocessor 505 or a packet engine 548. The flow distributors 550 cancomprise an interface mechanism that allows each flow distributor 550 tocommunicate with the other flow distributors 550 executing within thesystem 545. In one instance, the one or more flow distributors 550 candetermine how to balance load by communicating with each other. Thisprocess can operate substantially similarly to the process describedabove for submitting votes to an arbiter which then determines whichflow distributor 550 should receive the load. In other embodiments, afirst flow distributor 550′ can identify the load on an associated coreand determine whether to forward a first data packet to the associatedcore based on any of the following criteria: the load on the associatedcore is above a predetermined threshold; the load on the associated coreis below a predetermined threshold; the load on the associated core isless than the load on the other cores; or any other metric that can beused to determine where to forward data packets based in part on theamount of load on a processor.

The flow distributor 550 can distribute network traffic among the cores505 according to a distribution, computing or load balancing scheme suchas those described herein. In one embodiment, the flow distributor candistribute network traffic or; pad according to any one of a functionalparallelism distribution scheme 550, a data parallelism loaddistribution scheme 540, a flow-based data parallelism distributionscheme 520, or any combination of these distribution scheme or any loadbalancing scheme for distributing load among multiple processors. Theflow distributor 550 can therefore act as a load distributor by takingin data packets and distributing them across the processors according toan operative load balancing or distribution scheme. In one embodiment,the flow distributor 550 can comprise one or more operations, functionsor logic to determine how to distribute packers, work or loadaccordingly. In still other embodiments, the flow distributor 550 cancomprise one or more sub operations, functions or logic that canidentify a source address and a destination address associated with adata packet, and distribute packets accordingly.

In some embodiments, the flow distributor 550 can comprise areceive-side scaling (RSS) network driver, module 560 or any type andform of executable instructions which distribute data packets among theone or more cores 505. The RSS module 560 can comprise any combinationof hardware and software, In some embodiments, the RSS module 560 worksin conjunction with the flow distributor 550 to distribute data packetsacross the cores 505A-N or among multiple processors in amulti-processor network. The RSS module 560 can execute within the NIC552 in some embodiments, and in other embodiments can execute on any oneof the cores 505.

In some embodiments, the RSS module 560 uses the MICROSOFTreceive-side-scaling (RSS) scheme. In one embodiment, RSS is a MicrosoftScalable Networking initiative technology that enables receiveprocessing to be balanced across multiple processors in the system whilemaintaining in-order delivery of the data. The RSS may use any type andform of hashing scheme to determine a core or processor for processing anetwork packet.

The RSS module 560 can apply any type and form hash function such as theToeplitz hash function. The hash function may be applied to the hashtype or any the sequence of values. The hash function may be a securehash of any security level or is otherwise cryptographically secure. Thehash function may use a hash key. The size of the key is dependent uponthe hash function. For the Toeplitz hash, the size may be 40 bytes forIPv6 and 16 bytes for IPv4.

The hash function may be designed and constructed based on any one ormore criteria or design goals. In some embodiments, a hash function maybe used that provides an even distribution of hash result for differenthash inputs and different hash types, including TCP/IPv4, TCP/IPv6,IPv4, and IPv6 headers. In some embodiments, a hash function may be usedthat provides a hash result that is evenly distributed when a smallnumber of buckets are present (for example, two or four). In someembodiments, hash function may be used that provides a hash result thatis randomly distributed when a large number of buckets were present (forexample, 64 buckets). In some embodiments, the hash function isdetermined based on a level of computational or resource usage. In someembodiments, the hash function is determined based on ease or difficultyof implementing the hash in hardware. In some embodiments, the hashfunction is determined bases on the ease or difficulty of a maliciousremote host to send packets that would all hash to the same bucket.

The RSS may generate hashes from any type and form of input, such as asequence of values. This sequence of values can include any portion ofthe network packet, such as any header, field or payload of networkpacket, or portions thereof. In some embodiments, the input to the hashmay be referred to as a hash type and include any tuples of informationassociated with a network packet or data flow, such as any of thefollowing: a four tuple comprising at least two IP addresses and twoports; a four tuple comprising any four sets of values; a six tuple; atwo tuple; and/or any other sequence of numbers or values. The followingare example of hash types that may be used by RSS:

-   -   4-tuple of source TCP Port, source IP version 4 (IPv4) address,        destination TCP Port, and destination IPv4 address. This is the        only required hash type to support.    -   4-tuple of source TCP Port, source IP version 6 (IPv6) address,        destination TCP Port, and destination IPv6 address.    -   2-tuple of source IPv4 address, and destination IPv4 address.    -   2-tuple of source IPv6 address, and destination IPv6 address.    -   2-tuple of source IPv6 address, and destination IPv6 address,        including support for parsing IPv6 extension headers.

The hash result or any portion thereof may used to identify a core orentity, such as a packet engine or VIP, for distributing a networkpacket. In some embodiments, one or more hash bits or mask are appliedto the hash result. The hash bit or mask may be any number of bits orbytes. A NIC may support any number of bits, such as seven bits. Thenetwork stack may set the actual number of bits to be used duringinitialization. The number will be between 1 and 7, inclusive.

The hash result may be used to identify the core or entity via any typeand form of table, such as a bucket table or indirection table. In someembodiments, the number of hash-result bits are used to index into thetable. The range of the hash mask may effectively define the size of theindirection table. Any portion of the hash result or the hast resultitself may be used to index the indirection table. The values in thetable may identify any of the cores or processor, such as by a core orprocessor identifier. In some embodiments, all of the cores of themulti-core system are identified in the table. In other embodiments, aport of the cores of the multi-core system are identified in the table.The indirection table may comprise any number of buckets for example 2to 128 buckets that may be indexed by a hash mask. Each bucket maycomprise a range of index values that identify a core or processor. Insome embodiments, the flow controller and/or RSS module may rebalancethe network rebalance the network load by changing the indirectiontable.

In some embodiments, the multi-core system 575 does not include a RSSdriver or RSS module 560. In some of these embodiments, a softwaresteering module (Not Shown) or a software embodiment of the RSS modulewithin the system can operate in conjunction with or as part of the flowdistributor 550 to steer packets to cores 505 within the multi-coresystem 575.

The flow distributor 550, in some embodiments, executes within anymodule or program on the appliance 200, on any one of the cores 505 andon any one of the devices or components included within the multi-coresystem 575. In some embodiments, the flow distributor 550′ can executeon the first core 505A, while in other embodiments the flow distributor550″ can execute on the NIC 552. In still other embodiments, an instanceof the flow distributor 550′ can execute on each core 505 included inthe multi-core system 575. In this embodiment, each instance of the flowdistributor 550′ can communicate with other instances of the flowdistributor 550′ to forward packets back and forth across the cores 505.There exist situations where a response to a request packet may not beprocessed by the same core, i.e. the first core processes the requestwhile the second core processes the response. In these situations, theinstances of the flow distributor 550′ can intercept the packet andforward it to the desired or correct core 505, i.e. a flow distributorinstance 550′ can forward the response to the first core. Multipleinstances of the flow distributor 550′ can execute on any number ofcores 505 and any combination of cores 505.

The flow distributor may operate responsive to any one or more rules orpolicies. The rules may identify a core or packet processing engine toreceive a network packet, data or data flow. The rules may identify anytype and form of tuple information related to a network packet, such asa 4-tuple of source and destination IP address and source anddestination ports. Based on a received packet matching the tuplespecified by the rule, the flow distributor may forward the packet to acore or packet engine. In some embodiments, the packet is forwarded to acore via shared memory and/or core to core messaging.

Although FIG. 5B illustrates the flow distributor 550 as executingwithin the multi-core system 575, in some embodiments the flowdistributor 550 can execute on a computing device or appliance remotelylocated from the multi-core system 575. In such an embodiment, the flowdistributor 550 can communicate with the multi-core system 575 to takein data packets and distribute the packets across the one or more cores505. The flow distributor 550 can, in one embodiment, receive datapackets destined for the appliance 200, apply a distribution scheme tothe received data packets and distribute the data packets to the one ormore cores 505 of the multi-core system 575. In one embodiment, the flowdistributor 550 can be included in a router or other appliance such thatthe router can target particular cores 505 by altering meta dataassociated with each packet so that each packet is targeted towards asub-node of the multi-core system 575. In such an embodiment, CISCO'svn-tag mechanism can be used to alter or tag each packet with theappropriate meta data.

Illustrated in FIG. 5C is an embodiment of a multi-core system 575comprising one or more processing cores 505A-N. In brief overview, oneof the cores 505 can be designated as a control core 505A and can beused as a control plane 570 for the other cores 505. The other cores maybe secondary cores which operate in a data plane while the control coreprovides the control plane. The cores 505A-N may share a global cache580. While the control core provides a control plane, the other cores inthe multi-core system form or provide a data plane. These cores performdata processing functionality on network traffic while the controlprovides initialization, configuration and control of the multi-coresystem.

Further referring to FIG. 5C, and in more detail, the cores 505A-N aswell as the control core 505A can be any processor described herein.Furthermore, the cores 505A-N and the control core 505A can be anyprocessor able to function within the system 575 described in FIG. 5C.Still further, the cores 505A-N and the control core 505A can be anycore or group of cores described herein. The control core may be adifferent type of core or processor than the other cores. In someembodiments, the control may operate a different packet engine or have apacket engine configured differently than the packet engines of theother cores.

Any portion of the memory of each of the cores may be allocated to orused for a global cache that is shared by the cores. In brief overview,a predetermined percentage or predetermined amount of each of the memoryof each core may be used for the global cache. For example, 50% of eachmemory of each code may be dedicated or allocated to the shared globalcache. That is, in the illustrated embodiment, 2 GB of each coreexcluding the control plane core or core 1 may be used to form a 28 GBshared global cache. The configuration of the control plane such as viathe configuration services may determine the amount of memory used forthe shared global cache. In some embodiments, each core may provide adifferent amount of memory for use by the global cache. In otherembodiments, any one core may not provide any memory or use the globalcache. In some embodiments, any of the cores may also have a local cachein memory not allocated to the global shared memory. Each of the coresmay store any portion of network traffic to the global shared cache.Each of the cores may check the cache for any content to use in arequest or response. Any of the cores may obtain content from the globalshared cache to use in a data flow, request or response.

The global cache 580 can be any type and form of memory or storageelement, such as any memory or storage element described herein. In someembodiments, the cores 505 may have access to a predetermined amount ofmemory (i.e. 32 GB or any other memory amount commensurate with thesystem 575.) The global cache 580 can be allocated from thatpredetermined amount of memory while the rest of the available memorycan be allocated among the cores 505. In other embodiments, each core505 can have a predetermined amount of memory. The global cache 580 cancomprise an amount of the memory allocated to each core 505. This memoryamount can be measured in bytes, or can be measured as a percentage ofthe memory allocated to each core 505. Thus, the global cache 580 cancomprise 1 GB of memory from the memory associated with each core 505,or can comprise 20 percent or one-half of the memory associated witheach core 505. In some embodiments, only a portion of the cores 505provide memory to the global cache 580, while in other embodiments theglobal cache 580 can comprise memory not allocated to the cores 505.

Each core 505 can use the global cache 580 to store network traffic orcache data. In some embodiments, the packet engines of the core use theglobal cache to cache and use data stored by the plurality of packetengines. For example, the cache manager of FIG. 2A and cachefunctionality of FIG. 2B may use the global cache to share data foracceleration. For example, each of the packet engines may storeresponses, such as HTML data, to the global cache. Any of the cachemanagers operating on a core may access the global cache to servercaches responses to client requests.

In some embodiments, the cores 505 can use the global cache 580 to storea port allocation table which can be used to determine data flow basedin part on ports. In other embodiments, the cores 505 can use the globalcache 580 to store an address lookup table or any other table or listthat can be used by the flow distributor to determine where to directincoming and outgoing data packets. The cores 505 can, in someembodiments read from and write to cache 580, while in other embodimentsthe cores 505 can only read from or write to cache 580. The cores mayuse the global cache to perform core to core communications.

The global cache 580 may be sectioned into individual memory sectionswhere each section can be dedicated to a particular core 505. In oneembodiment, the control core 505A can receive a greater amount ofavailable cache, while the other cores 505 can receiving varying amountsor access to the global cache 580.

In some embodiments, the system 575 can comprise a control core 505A.While FIG. 5C illustrates core 1 505A as the control core, the controlcore can be any core within the appliance 200 or multi-core system.Further, while only a single control core is depicted, the system 575can comprise one or more control cores each having a level of controlover the system. In some embodiments, one or more control cores can eachcontrol a particular aspect of the system 575. For example, one core cancontrol deciding which distribution scheme to use, while another corecan determine the size of the global cache 580.

The control plane of the multi-core system may be the designation andconfiguration of a core as the dedicated management core or as a mastercore. This control plane core may provide control, management andcoordination of operation and functionality the plurality of cores inthe multi-core system. This control plane core may provide control,management and coordination of allocation and use of memory of thesystem among the plurality of cores in the multi-core system, includinginitialization and configuration of the same. In some embodiments, thecontrol plane includes the flow distributor for controlling theassignment of data flows to cores and the distribution of networkpackets to cores based on data flows. In some embodiments, the controlplane core runs a packet engine and in other embodiments, the controlplane core is dedicated to management and control of the other cores ofthe system.

The control core 505A can exercise a level of control over the othercores 505 such as determining how much memory should be allocated toeach core 505 or determining which core 505 should be assigned to handlea particular function or hardware/software entity. The control core505A, in some embodiments, can exercise control over those cores 505within the control plan 570. Thus, there can exist processors outside ofthe control plane 570 which are not controlled by the control core 505A.Determining the boundaries of the control plane 570 can includemaintaining, by the control core 505A or agent executing within thesystem 575, a list of those cores 505 controlled by the control core505A. The control core 505A can control any of the following:initialization of a core; determining when a core is unavailable;re-distributing load to other cores 505 when one core fails; determiningwhich distribution scheme to implement; determining which core shouldreceive network traffic; determining how much cache should be allocatedto each core; determining whether to assign a particular function orelement to a particular core; determining whether to permit cores tocommunicate with one another; determining the size of the global cache580; and any other determination of a function, configuration oroperation of the cores within the system 575.

F. Systems and Methods for Distributing Data Packets Across a Multi-CoreArchitecture and System

1. Multi-Core System and Architecture for Distributing Data PacketsAcross a Multi-Core Architecture and System

The system and architecture described in FIG. 5B is a general overviewof one possible multi-core system 545 that can accomplish thesymmetrical distribution of requests and responses across packet enginesexecuting on multiple cores in a multi-core system 545. There exist anumber of additional system aspects that can, in some embodiments,facilitate the symmetrical distribution of requests and responses andthat can further implement security policies and other systemconfigurations that require the maintenance of a client IP address or aclient port number. In systems where the multi-core architecture 545processes fragmented requests and/or responses, additional objects andstructures are needed to process and track fragmented data packets.

Illustrated in FIG. 6A is an embodiment of a multi-core system 545. Thissystem 545 can include, in most embodiments, one or more networkinterface cards (NIC) 552 which can execute or include a RSS module 560.The NIC 552 can communicate with one or more cores 505 where each corecan execute a packet engine 548 and/or a flow distributor 550. In someembodiments, each core 505 can store one or more port allocation tables604 and can comprise one or more ports 632 and one or more internetprotocol (IP) addresses 630.

Further referring to FIG. 6A and in more detail, in one embodiment, themulti-core system 545 can be any of the multi-core systems 545 describedherein. In particular, the multi-core system 545 can be any of themulti-core systems 545 described in FIGS. 5B-5C. The multi-core system545 can execute on an appliance 200, a client, a server or any othercomputing machine that executes the multi-core system 545 describedherein. While the multi-core system 545 illustrated in FIG. 6A includesa plurality of cores 505 and a NIC 552, in some embodiments themulti-core system 545 can comprise additional devices and can executeadditional programs, clients and modules.

In one embodiment, the multi-core system 545 can comprise a NIC 552 suchas any of the NICs described herein. Although the multi-core system 545illustrated in FIG. 6A depicts a multi-core system 545 having a singleNIC 552, in some embodiments, the multi-core system 545 can have aplurality of NICs 552. These NICs 552 can be the same type of NIC 552,and in other embodiments can be different types of NICs 552. The NIC(s)552 can communicate with one or more of the processing cores 505 in themulti-core system 545. For example, the NIC 552 can communicate witheach of a first core 505A, a second core 505B, a third core 505C, afourth core 505D, a fifth core 505E, a sixth core 505F, a seventh core505G, and any “N” number of cores 505N, where “N” is a whole numbergreater than zero. In other embodiments, the NIC 552 can communicatewith a single core 505 or a subset of cores 505. For example, the NIC552 may communicate with a first core 505A, or cores one through 4505A-505D. In embodiments where multiple NICs 552 are included withinthe multi-core system 545, each NIC 552 can communicate with one or morecores 505. For example, a first NIC 552 can communicate with cores onethrough 4 505A-505D, while a second NIC 552 can communicate with coresfive through seven 505E-505G. In other embodiments where multiple NICs552 are included within the multi-core system 545, one or more NICs 552can communicate with the cores 505 while the other NICs 552 can performan alternative function, communication with other systems or deviceswithin the multi-core system 545, or can function as redundant NICs 552that are used as backup when a primary NIC 552 fails. The NICs 552, insome embodiments, can interface with a network and with the multi-coresystem 545 via transmit and receive queues without having specificknowledge of the cores 505 or of the multi-core system 545 architecture.In these embodiments, the NICs 552 can simply transmit data packetsstored in a NIC transmit queue, and receive network packets transmittedover a network.

In some embodiments, the NIC 552 executes a RSS module 560 such as anyof the RSS module 560 described herein. The RSS module 560 applies ahash function to a tuple or sequence of values comprising anycombination of the following: a client IP address; a client port; adestination IP address; a destination port; or any other valueassociated with the source or destination of a data packet. In someembodiments, the value that results from the application of the hashfunction to the tuple, identifies a core 505 within the multi-coresystem 545. The RSS module 560 can use this property of the hashfunction to distribute packets across cores 505 in a multi-core system545. By distributing packets across the cores 505 of the multi-coresystem 545, the RSS module 560 can symmetrically distribute networktraffic across the cores 505 in a manner substantially similar toflow-based data parallelism.

The cores 505 within the multi-core system 545 can be any of the cores505 described herein. In one embodiment, the multi-core system 545 caninclude any “N” number of cores where “N” is a whole number greater thanzero. In other embodiments, the multi-core system 545 can include eightcores. Cores 505 can be dedicated to process programs or servicesperforming certain functions, and in some embodiments, can be dedicatedto process data packets received or transmitted by certain devices orprogram modules. In some embodiments, each core 505 can execute any ofthe following: a packet engine 548 such as any of the packet engines 548described herein or a flow distributor 550 such as any of the flowdistributors 550 described herein. In other embodiments, each core 505stores, in an associated storage repository, any of the following: aport allocation table; a listing of ports of the core 505; or a listingof IP addresses of the core 505.

In one embodiment, each core 505 executes a packet engine 548A-N and mayinclude any of the vServers 275 described herein. A packet engine 548A-Ncan be included in each core 505, and collectively the packet engines548A-N can be referred to as a packet engine 548. Packet engines 548, insome embodiments, alter or modify tuples of data packets according toflow distribution rules executed by each packet engine 548. In oneembodiment, a packet engine 548 replaces a client IP address in a tupleof a data packet received by the packet engine 548, with an IP address630A-B of the core 505 on which the packet engine 548 executes. Thepacket engine 548, in another embodiment, replaces a client port in atuple of a data packet received by the packet engine 548, with a port632A-B selected from a plurality of ports 632A-B of the core 505 onwhich the packet engine 548 executes. In still other embodiments, thepacket engine 548 maintains all aspects of a data packet including thecontents of a tuple of the data packet. The packet engine 548, in someembodiments, communicates with one or more servers 106 to forwardservers 106 received data packets that are destined for those servers106. Similarly, the packet engine 548, in some embodiments, communicateswith one or more clients 102 to forward clients 102 received datapackets that are destined for those clients 102.

Each core 505, in some embodiments, accesses a storage repositoryallocated to each core 505 or a shared storage repository available toall cores 505 in a multi-core system 545 via a packet engine 548 or anyother module or object executing on the core 505. Thus, each module,program, client and/or object executing on the core 505 can access anystorage repository accessible to the core 505. In one embodiment, a portallocation 604A-N is stored in a storage repository either shared orallocated to a specific core 505. A single core 505 can have one or moreport allocation tables 604A-N (referred to generally as port allocationtable 604,) where each port allocation table 604 lists both availableand un-available ports on a particular core 505A. In one embodiment, acore 505 can have one port allocation table 604, while in otherembodiments a core 505 can have 64 or 256 port allocation tables 604.For example, Port Allocation Table A 604A on Core 1 505A can storeentries indicating the status of each port 632A-B on Core 1 505A. Thestatus of each port 632A-B can include any of the followingcharacteristics: whether the port is open or closed; whether the porthas been assigned, i.e. whether the port is available or un-available;whether the port is within a pre-assigned range; and any other pertinentcharacteristic of the port. Thus, if Packet Engine A 548A on Core 1 505Awants to determine whether a particular port is open and/or available,Packet Engine A 548A can query Port Allocation Table A 604A to determinewhether the desired port is open and/or available.

In instances where a core 505 has multiple port allocation tables 604,each port allocation table can be associated with a value or otherunique identifier. Each port allocation table 604, in one embodiment,has an identifying value that can be determined by applying a hashfunction to a portion of a tuple of a data packet. Therefore any of thehashes described herein can be applied by a packet engine 548 or flowdistributor 550 to any combination of a client IP address, a clientport, a destination IP address and/or a destination port to determine aunique value for that data packet. This unique value further identifiesa port allocation table 604 on the core 505. For example, if a packetengine 548B on Core 2 505B wants to assign a port to a received datapacket, the packet engine 548B first applies a hash to a client IPaddress and a destination IP address identified in the data packet.Based on the result of the hash, the packet engine 548B selects a portallocation table 604 from amongst one or more port allocation tables 604on Core 2 505B, and selects a port 632C-D based on a review of theselected port allocation table 604.

Port allocation tables 604, in some embodiments, can be dynamicallyaltered by a packet engine 548, flow distributor 550 or other program,service or device based on changes made to ports 632 of a core 505, orbased on allocation of ports 632 to a data packet or transaction. In oneembodiment, when a section of ports is assigned to a particular portallocation table 604 in a core 505 or to a particular core 505, the portallocation table 604 is updated to reflect the assignment. The updatecan either be an update to the entries of the affected ports 632 toreflect the assignment, or an update of the affected ports 632 to listthe ports 632 within the section of ports 632 as open and all otherports 632 as closed. In other embodiments, once a port is assigned to adata packet or transaction between two computing machines, the portallocation table 604 is updated to reflect the assignment by listing astate or status of the assigned port, such as closed or unavailable andin some cases by identifying the data packet or transaction.

In some embodiments, each packet engine 548 or core 505 can be assigned,allocated or otherwise associated with one or more port numbers 632(referred to generally as ports 632.) A port number can be a logicaldata structure for an endpoint in a network, and in some embodiments canbe referred to as a port. In some embodiments, a port number can beincluded in the header of a data packet and can refer to a process thatthe data packet is to be forwarded to. While FIG. 6A illustrates eachcore 505 as having two ports 632, each core 505 has multiple ports 632,i.e. hundreds and in some cases thousands or millions of ports 632.Ports 632, in most embodiments, are identified by unique values ornumbers. Assigning a data packet or transaction to a port 632 cancomprise updating a header of the data packet or data packets of thetransaction to reflect the unique value or number associated with theassigned port 632. Ports 632, in many embodiments, are tracked withinport allocation tables 604 on each core 505. While each core 505 has itsown set of ports 632, the values or number associated with each port 632can repeat on each core 505. For example, Core 3 505C can have ports onethrough three-thousand, while Core 5 505E can also have ports onethrough three-thousand. The uniqueness of each port in Core 3 505C andCore 5 505 E comes from the fact that Core 3 505C ports are associatedwith one or more IP addresses specific to Core 3 505C, and Core 5 505Eports are associated with one or more IP addresses specific to Core 5505E.

Similarly, each packet engine 548 or core 505 can be assigned,allocated, associated with or host one or more IP addresses 630A-B.While FIG. 6A illustrates each core 505 as having two IP addresses 630(referred to generally as IP addresses 630,) each core 505 can have any“N” number of IP addresses 630 where “N” is a whole number greater thanzero. In some embodiments, the IP addresses 630 of a core 505 arepre-assigned by an administrator, application or other service orprogram executing in the multi-core system 545. In other embodiments, agroup or range of IP addresses 630 are assigned to each core 505. Instill other embodiments, the same IP address 630 is assigned to eachcore 505. This IP address 630, in most embodiments, is an IP address ofthe multi-core system 545.

In one embodiment, a first core 505 can execute a flow distributor 550.The flow distributor 550 can be any of the flow distributors 550described herein. While FIG. 6A illustrates a multi-core system 545where the flow distributor 550 executes on a first core 505, each core505 can execute an instance of the flow distributor 550 specific to thatcore 505. In instances where the flow distributor 550 executes on asingle core 505, that core can be considered the control or master core.In still other embodiments, the flow distributor 550 can executed on atleast one NIC 552 in the multi-core system 545. In embodiments where aRSS module 560 is included in the multi-core system 545, the system 545may not include a flow distributor 550.

Illustrate in FIG. 6B is a detailed description of at least one of thecores 505 in a multi-core system 545. The core 505N can be any of the“N” cores in the multi-core system 545 where “N” is a whole numbergreater than zero. The core 505N can comprise a flow distributor 550, apacket engine 548N, one or more port allocation tables 604, and one ormore IP addresses 630. The packet engine 548N can execute afragmentation module 650 that can further access a fragmentation table655 accessible by both the packet engine 548N and the fragmentationmodule 650. Each port allocation table 604 can store or track one ormore ports 632.

Further referring to FIG. 6B, and in more detail, in one embodiment themulti-core system 545 can be any of the above-described multi-coresystems 545. Similarly, the core 505 can be any of the above-describedcores 505. In one embodiment, each of the cores 505 in the multi-coresystem 545 comprises the elements of the core 505 described in FIG. 6B.In other embodiments, the cores 505 of the multi-core system 545comprise combinations of the elements of the core 505 described in FIG.6B.

In one embodiment, the core 505 can execute a flow distributor 550 or aninstance of a flow distributor 550. In some embodiments, the core 505can execute multiple instances of a flow distributor 550. The flowdistributor 550 can be any of the flow distributors 550 describedherein. In other embodiments, the core 505 does not execute or otherwisecomprise a flow distributor 550 or an instance of a flow distributor550. The core 505, in these embodiments, can communicate with a flowdistributor 550 executing on another core 505 or on another device inthe multi-core system 545 via the packet engine 548N or another programor module executing on the core 505.

The core 505 or a packet engine 548 executing on the core 505 can accessor otherwise be associated with multiple port allocation tables 604,supra. In one embodiment, the core 505 can access a single portallocation table, while in other embodiments the core 505 can access an“N” number of port allocation tables where “N” is a whole number greaterthan zero. The port allocation table 604 can be any of the portallocation tables 604 described herein. While FIGS. 6A-6B describe aport allocation table, in other embodiments each core 505 can access aport list of available and un-available ports. In still otherembodiments, each core 505 can access a storage repository storinginformation about the availability of each port 632 of a core 505.

Port allocation tables 604, in most embodiments, track characteristicsor the status of ports 632 of or used by a core 505. A port allocationtable 604 can track which ports are available, open or free on all localIP addresses of a core 505 or of the multi-core system 545. The ports632, in many embodiments, can be any of the ports described herein, andcan be any port. In some embodiments, ports 632 are associated with aparticular port allocation table 604. For example, Port Allocation TableA 604A tracks ports 1-N 632A-N, while Port Allocation Table B 604Btracks ports 1-N 632A-N. In each case, the ports 632 tracked by the portallocation table are specific to that port allocation table. Therefore,although the ports 632 may be the same number, the ports 632 tracked byPort Allocation Table A 604A are specific to Port Allocation Table A604A, and the ports 632 tracked by Port Allocation Table B 604B arespecific to Port Allocation Table B 604B. The specificity of each portis determined by the characteristics of the tuple of the data packet towhich a port is allocated. For example, a first data packet has a firsttuple with a first client IP address and a first destination address. Asecond data packet has a second tuple different than the first tuple andcontaining either or both of a different client IP address anddestination address, i.e. a second client IP address and a seconddestination address. Although each of first data packet and the seconddata packet may be assigned the same port number; the first data packetcan be associated with a port allocation table 604 corresponding to thefirst client IP address and/or the first destination address. Similarly,the second data packet can be associated with a port allocation table604 corresponding to the second client IP address and/or the seconddestination address.

In some embodiments, the port allocation table 604 or a portion of theport allocation tables can be stored on a computing device or in astorage repository remotely located from the multi-core system 545. Theport allocation table(s) 604 can be stored on an appliance, computingmachine, or in a storage repository located outside of the multi-coresystem 545. When the port allocation table(s) 604 are located outside ofthe multi-core system 545, the computing machine, device or a program oragent executing on the computing machine, device or within the storagerepository can communicate with the multi-core system 545. Oncecommunication between the remote port allocation table(s) 604 and themulti-core system 545 is established, packet engine(s) 548 in themulti-core system 545 can query and update the remote port allocationtable(s) 604 in a manner substantially similar to the manner in whichthe packet engine(s) 548 queries and updates local port allocationtable(s) 604.

Each core 505 in the multi-core system 545, in some embodiments,includes one or more IP addresses 630A-N (generally referred to as IPaddress 630.) The IP addresses 630 can be any IP address or address, andcan be any IP address 630 described herein. In one embodiment, each portallocation table 604 can be associated with a particular IP address 630.This IP address 630, in some embodiments, can be a proxy or dummy IPaddress such as 0.0.0.1. Similarly, in some embodiments, the cores 505of the multi-core system 545 can be associated with a particular IPaddress 630 or range of IP addresses.

In some embodiments, the packet engine 548N executes or comprises afragmentation module 650. The fragmentation module 650, in someembodiments, can be a hardware element included in the multi-core system545. In other embodiments, the fragmentation module 650 is a softwaremodule executing on the core 505. In still other embodiments, the packetengine 548 executes the fragmentation module 650 which can comprise anycombination of hardware and software. The fragmentation module 650 canbe, in some embodiments, included in the packet engine 548 such that thepacket engine 548 executes instructions that are otherwise executed bythe fragmentation module 650. Additionally, in some embodiments, thepacket engine 548N can access a fragmentation table 655 stored in memorywithin the multi-core system 545. The fragmentation module 650, in someembodiments, inputs data packet fragments and applies a fragmentationaction. In embodiments where the fragmentation action is “Assemble,” thefragmentation module 650 assembles the data packet fragments tore-generate or recreate the data packet. In other embodiments where thefragmentation action is “Bridge,” the fragmentation module 650 transmitseach data packet fragment to a different core 505 for re-assembly intothe original data packet. In some embodiments, the fragmentation module650 assembles the data packet fragments to re-generate or recreate thedata packet regardless of whether the fragmentation action is “Assemble”or “Bridge.” The fragmentation action, in some embodiments, can dictateany of the following: assemble a port of the data packet fragments andbridge the remaining fragments; mark the data packet fragments prior tobridging them; assemble only those data packet fragments having apre-determined set of characteristics; assemble only the header of thedata packet and transmit the rest of the data packet fragments to adifferent core 505 for re-assembly.

In one embodiment, the fragmentation module 650 determines afragmentation action based in part on whether a protocol control block(PCB) or network address translation protocol control block (NATPCB) iscreated. When either of a PCB or NATPCB is present, the packet engine548 or the flow distributor 550 that receives a fragmented data packetfirst determines a destination core for the data packet fragments. Thefragmentation action to be applied to the data packet fragments can bedetermined based in part on the type of connection between themulti-core system 545 and the computing machine that originated the datapacket fragments. In some embodiments, determining the fragmentationaction comprises doing PCB, NATPCB, fragmentation rule, Reverse NAT(RNAT) and service lookups. The packet engine or flow distributor thatreceives the data packet fragments, in one embodiment, forwards thedetermined fragmentation action to a packet engine or flow distributorexecuting on the destination core. Thus, when the data packet fragmentsare transmitted to the destination core, the fragmentation action can beapplied to the data packet fragments.

In other embodiments, when either of a PCB or NATPCB is present, thepacket engine 548 or the flow distributor 550 that receives a fragmenteddata packet first assembles the data packet fragments into a reassembleddata packet until a complete packet header is available. A destinationcore for the data packet fragments is then determined. If the core thatreceived the data packet fragments is not the destination core, then apacket engine or flow distributor on the receiving core does NATPCB/PCBlookups until a fragmentation action is determined. In embodiments wherethe receiving core is the destination core, a packet engine on thereceiving core does service and RNAT lookups to determine thefragmentation action.

In many embodiments, when a receiving core is not the destination core,the receiving core can determine the fragmentation action and transmit amessage to the destination core indicating the correct fragmentationaction. In one embodiment, a packet engine on the receiving coretransmits the fragmentation action along with the following values: asource IP address; a destination IP address; a source port and adestination port. The fragmentation action when determined can be storedin a fragmentation table 655. In some embodiments, when a destinationcore receives a fragmentation action a packet engine or flow distributoron the destination core can store the fragmentation action in afragmentation table 655. The fragmentation action can be stored alongwith any of the following identifying information: a client IP address;a source IP address; a destination IP address; a source port; a clientport; or a destination port.

In some embodiments, when the received data packet fragments are UDPfragments, each data packet is hashed based on a two tuple. This twotuple can comprise any of the following values: a client IP address; asource IP address; a destination IP address; a source port; a clientport; or a destination port. A determination about the fragmentationaction and a determination as to what the destination core is can bemade according to any of the above-described methods.

Distribution of data packets, network traffic or requests and responsescan be accomplished by any of the parallel computing schemes describedherein. In one embodiment, the distribution of network traffic can bebased on a symmetric flow distribution. Symmetric flow distribution canbe accomplished using the Toeplitz hash or any comparable hash todetermine a destination core for each data packet received by themulti-core system 545. In some embodiments, the symmetric flowdistribution hash, or the symmetric hash distribution (SHD) has issubstantially the same as the hash used by the RSS module 560. The hashoperates by inputting a byte stream, such as a tuple or sequence ofvalues, and supplying the RSS driver within the RSS module 560 with akey that can be used within the hash calculation. Thus, when an array of“N” bytes is inputted into the hash function, the byte stream can beidentified as input[0] input[1] input[2] input[N−1]; where the leftmostbyte is input[0] and the leftmost bit is the most significant bit ofinput [0], and where the rightmost byte is input [N−1] and the rightmostbit is the least significant bit of input [N−1]. The hash can, in someembodiments, operate according to the following relationship:

-   -   For all inputs up to the value “N”, calculate the following: for        each bit “B” in input[ ] from left to right, if “B” is equal to        one then (“Result”̂=(leftmost 32 bits of K)) and shift K left 1        bit position, then return “Result.”        The hash, in some embodiments, is distributed over a XOR        operation according to the following equation or relationship,        Hash(A xor B)=Hash(A) xor Hash(B). In other embodiments, the        hash can be distributed over any logical operation such as:        NAND; NOR, OR, AND or any other logical operation functional in        the methods and systems described herein.

The tuple or sequence of values inputted into the hash can be aconcatenation of any of the following values: client IP address; sourceIP address; destination IP address; local IP address; dummy IP address;assigned IP address; appliance IP address; client port; source port;destination port; local port; dummy port; assigned port; appliance port;or any other IP address or port. In some embodiments, the order of thetuple is maintained such that the tuple is a concatenation of client IPaddress, client port, destination IP address and destination port. Thetuple can comprise two, four, six or any number of values. Additionally,the tuple can comprise any type of value, i.e. numeric, binary, trinary,alphabetic, or alpha-numeric.

Included below are examples of how the hash is applied in differentversions of the internet protocol and when TCP or UDP is used. Theseexamples are meant to be illustrative of applying the hash and are notmeant to limit the scope of the

Example 1 IPV4: TCP/UDP

In this example, the tuple comprises a concatenation of the followingvalues: source address; destination address; source port; anddestination port. The tuple, or input string, can therefore becharacterized by the following relationship: INPUT[12]=@12-15, @16-19,@20-21, @22-23. The entries @n-m identify a byte range, i.e. n=12, m=15,@12-15. The application of the hash to this input string ischaracterized by following equation:

Hash Result=ComputeHash(Input,12)

Example 2 IPV4: Others

In this example, the tuple comprises a concatenation of the followingvalues: source address; and destination address. The tuple, or inputstring, can therefore be characterized by the following relationship:INPUT[8]=@12-15, @16-19. The entries @n-m identify a byte range, i.e.n=12, m=15, @12-15. The application of the hash to this input string ischaracterized by following equation:

Hash Result=ComputeHash(Input,8)

Example 3 IPV6: TCP/UDP

In this example, the tuple comprises a concatenation of the followingvalues: source address; destination address; source port; anddestination port. The tuple, or input string, can therefore becharacterized by the following relationship: INPUT[36]=@8-23, @24-39,@40-41, @42-43. The entries @n-m identify a byte range, i.e. n=8, m=23,@8-23. The application of the hash to this input string is characterizedby following equation:

Hash Result=ComputeHash(Input,36)

Example 4 IPV6: Others

In this example, the tuple comprises a concatenation of the followingvalues: source address; and destination address. The tuple, or inputstring, can therefore be characterized by the following relationship:INPUT[32]=@8-23, @24-39. The entries @n-m identify a byte range, i.e.n=8, m=23, @8-23. The application of the hash to this input string ischaracterized by following equation:

Hash Result=ComputeHash(Input,32)

In some embodiments, when the multi-core system 545 intercepts orotherwise processes data packets and/or network traffic that does notuse the internet protocol, no hash is calculated. In this embodiment,the non-IP packets or traffic can be routed to a default core 505. Thiscore 505 can be dedicated to handling non-IP packets or can allocate acertain amount of resources to the handling and processing of non-IPnetwork traffic.

2. A Method and System for Providing Symmetrical Request and ResponseProcessing

Distributing network traffic amongst one or more cores in a multi-coresystem, can include obtaining a data packet or request, identifying atuple of the data packet, applying a hash to the tuple, a forwarding thedata packet to a core identified by the hash result. The hash can be anyof the above mentioned hashes, or can be any hash having thecharacteristics of the above-described hash. In particular, the hash canbe any hash that, when applied to a tuple, generates a result thatidentifies at least one core in a multi-core system. The tuple can becomprised of any number of characteristics of the data packet. In someembodiments, the tuple can comprise a source IP address, a destinationIP address, a source port and a destination port.

To ensure that a response or other data packet associated with thepreviously processed data packet is forwarded or otherwise distributedto the same core, the packet engine selects an IP address of the packetengine or core, and a port number of the packet engine or core thattogether with the destination IP address and destination port cancomprise a second tuple. The result of applying the above-described hashto the second tuple is a hash result that identifies the first core. Bymodifying the request with the second tuple, the packet engine canensure that any response to the request will include the second tuple.Accordingly, when a flow distributor applies the above-described hash toa tuple of the response, the result will identify the first core. Thus,the flow distributor distributes the response to the same first corethat the request was distributed to.

Each tuple is sufficiently unique such that the hash result is alsosufficiently unique for the purposes of symmetrically distributingrequests and responses across one or more packet engines executing oncores within the multi-core system 545. The hash is symmetric becausethere can exist another tuple that will cause the hash to generateexactly the same result as the result of applying the hash to a firsttuple. To verify that the second tuple will cause a response packet toreturn to the same core, the packet engine chooses the elements of thesecond tuple to ensure that they will generate a hash resultsubstantially the same as a first hash resulting from the application ofthe hash to the first tuple.

Illustrated in FIG. 7A is a flow diagram depicting one embodiment of amethod 700 for using the above-discussed hash to distribute networktraffic amongst one or more cores 505 in a multi-core system 545. First,a flow distributor 550 or RSS module 560 of the multi-core system 545,receives data packets from a client, server or other computing machine(Step 704), and calculates a hash value by applying the hash to a firsttuple of the received data packet (Step 706). The first tuple cancomprise a client IP address, a destination IP address, a client port,and a destination port. Applying the hash to the first tuple can, insome embodiments, result in a value sometimes referred to as the hash. Acore 505 in the multi-core system 545 is selected based on the hashresult value (Step 708) and the received data packet is forwarded to theselected core (Step 710). At this point the first tuple still comprisesthe following values: client IP address; destination IP address; clientport; and destination port. A packet engine 548 on the selected core 505receives the data packet and updates the tuple with a selected IPaddress of either the multi-core system 545, appliance 200, or selectedcore 505 (Step 712). The first tuple now comprises the following values:the selected IP address; the destination address; the client port; andthe destination port. The packet engine 548 can then identify a portthat, when included in the first tuple in lieu of the client port, willcause the data packet the return to the selected core 505. Uponidentifying this port, the packet engine 548 updates the first tuplewith the selected port (Step 714). The elements of the first tuple nowcomprise: the selected IP address; the destination address; the selectedport; and the destination port. The data packet and its modified tupleare then transmitted to a server, client or other computing machine(Step 716). Any responses to this data packet are forwarded to andreceived by the multi-core system 545 (Step 704). The method 700 thenrepeats itself.

Further referring to FIG. 7A, in one embodiment the client IP addressand the client port can refer to a source IP address and a source port.The source IP address identifies the computing machine or appliance fromwhich the data packet originated. In some embodiments, the sourcecomputing machine or appliance generated the data packet. In oneembodiment the client IP address can refer to a client, while in otherembodiments the client IP address can refer to a server or othercomputing machine or appliance. Similarly, the destination IP addressidentifies a destination computing machine or appliance to which thedata packet is being transmitted. In some embodiments the destinationcomputing machine or appliance is a server, while in other embodimentsthe destination computing machine or appliance is a client or othercomputing machine or appliance.

In some embodiments steps of the method 700 are carried out by a flowdistributor 550. In other embodiments, these steps can be carried out bya RSS module 560. In still other embodiments, these steps can be carriedout by a combination of a RSS module 560 and flow distributor 550. Inother embodiments, the flow distributor 550 is used when the NIC 552 isa RSS-unaware NIC 552, i.e. the NIC 552 does not include a RSS module560. In still other embodiments, another distribution module or clientexecuting within the multi-core system 545 can carry out any of theactions or steps carried out by the flow distributor.

Data packets received from a client, in some embodiments, are requests.In other embodiments, data packets received from a client areinformation, responses, updates, or any other type of information orcommunication. Data packets received from a server, in some embodiments,are responses. In other embodiments, data packets received from a serverare information, requests updates or any other type of information orcommunication.

In many embodiments, the multi-core system 545 receives data packetsfrom clients and/or servers on a network 104 (Step 704). The multi-coresystem 545, in most embodiments, is installed in front of one or moreservers, clients and other computing machines and appliances such thatany data packets transmitted to or by these servers, clients and othercomputing machines and appliances, must pass through the multi-coresystem 545. Thus, in some embodiments a NIC 552 in the multi-core system545 receives all data packets. In other embodiments, one or more NICs552 in the multi-core system 545 receive each data packet transmitted toor by the servers, clients and computing machines. The flow distributor550 of the multi-core system 545 drains or otherwise obtains thereceived data packets from a NIC 552 receive queue in the NIC 552. Uponobtaining a data packet from the NIC 552 receive queue, the flowdistributor 550 determines to which core 505 in the multi-core system545 the data packet should be sent.

At the time the flow distributor 550 obtains a data packet from the NIC552 receive queue, the data packet has a series of values that togethercomprise a tuple. In some embodiments, this tuple, or series of values,comprises a client IP address, a destination IP address, a client portand a destination port. The client IP address is the IP address of thesource of the data packet, which in some instances can be a client andin other instances can be a server or other computing machine. Thedestination IP address is the IP address of the computing machine orappliance to which the data packet is being sent. Thus, in someinstances the destination IP address is an address of a server and inother embodiments the destination IP address is an address of a client.The client port and the destination port are ports associated witheither the source machine or the destination machine. These ports aretypically configured prior to sending the data packet, however in someembodiments, the client port and/or the destination port is a dummy orproxy port, while in other embodiments the client port and/or thedestination port is a default port.

Once the multi-core system 545 receives the data packet, the flowdistributor 550 or any other module or program executing within themulti-core system 545 can apply the above-described hash to the firsttuple (Step 706). In some embodiments, the first tuple is created priorto applying the hash. The first tuple can be created by concatenatingthe client IP address, the destination IP address, a client port and asource port. These values, in some embodiments, are stored in a headerin the data packet. In other embodiments, these values are stored inmetadata associated with the data packet. In still other embodiments,these values are stored in the load portion of the data packet and mustbe extracted from the data packet prior to creating the tuple. In someembodiments, concatenating these values can be done by any one of theRSS module 560, the flow distributor 550, or a concatenation program ormodule executing in the multi-core system 545. In other embodiments,concatenating these values can occur as part of the hash. In someembodiments, the hash can be applied according to any of theabove-described methods. Applying the hash, in many cases, results inoutput such as a result value, a hash value or any other valuerepresentative of the outcome of applying the hash to the first tuple.

While the hash can, in some embodiments, be calculated by the flowdistributor 550 or another module executing within the multi-core system545, in other embodiments the hash can be calculated by a computingmachine or appliance outside of the multi-core system 545. In oneembodiment, a router remotely located outside of the multi-core system545 can intercept data packets before they are received by themulti-core system 545. In this embodiment, the router can apply the hashto the data packets to determine which core 505 in the multi-core system545 should receive each data packet. After determining to which core 505a particular data packet should be transmitted, the router can transmitthe data packet to the multi-core system 545 addresses in such a mannerthat the multi-core system 545 forwards the data packet to the propercore 505. In other embodiments, the hash can be applied by a computingmachine or different appliance.

The flow distributor 550 or RSS module 560 can, in some embodiments,select a core 505 from the multi-core system 545 (Step 708) based on avalue resulting from the application of the hash to the first tuple. Insome embodiments, the value generated by the hash points to oridentifies a core 505 in the multi-core system 545. This property of thehash can be exploited to substantially evenly distribute network trafficamongst the cores 505 in the multi-core system 545. In one embodiment, atable storing a listing of possible hash values and their correspondingcores is stored in a memory element or storage repository within themulti-core system 545. The flow distributor 550 or RSS module 560, uponapplying the hash to obtain a resultant value, can query the table for acore corresponding to the resultant hash value. Entries in the table canbe designed to ensure even distribution of network traffic amongst thecores 505.

Upon selecting a core 505, the data packet is forwarded to the selectedcore 505 (Step 710). The data packet can be forwarded by any one of theflow distributor 550, the RSS module 560 or an intra-core communicator(Not Shown.) In some embodiments, forwarding the data packet can includecopying the data packet into a memory element, storage repository orcache that is accessible by each of the cores 505 in the multi-coresystem 545; and forwarding the core 505 selected to receive the datapacket, a message indicating that the data packet is stored in memoryand available for download by or to the selected core 505. A packetengine 548 or other module executing on the selected core 505 could thenaccess the shared memory element to download the data packet. In otherembodiments, the data packet can be forwarded to the selected core 505via a core-to-core messaging system that uses an internal networkcomprising each of the cores 505 in the multi-core system 545. Thiscore-to-core messaging system can utilize a network internal to themulti-core system 545 and addresses specific to each core 505 or packetengine 548 within the multi-core system 545. In some embodiments, datapackets can be transmitted to a destination address of the core-to-coremessaging system that corresponds to the selected core 505.

When a data packet is forwarded or transmitted to a selected core 505(Step 710), the data packet can be received by a packet engine 548executing on the selected core 505. Packet engines 548, in someembodiments, manage the receipt and transmission of data packetsforwarded to cores 505. Once the packet engine 548 receives the datapacket, the packet engine 548 can make any number of determinationsabout the data packet and can perform any number of operations on thedata packet. In one embodiment, the packet engine 548 can determine thatthe source IP address and the source port of the first tuple does nothave to be maintained. Based on this determination, the packet engine548 can modify the first tuple to include a different source IP addressand/or a different source port.

When a determination is made that packet engine 548 can modify either orboth the client IP address and the client port, the packet engine 548can then replace the client IP address with an IP address of theselected core 505 (Step 712). In some embodiments, the IP address can bean IP address of the multi-core system 545. In other embodiments, the IPaddress can be an IP address of the appliance 200. In still otherembodiments, the IP address can be any one of the IP addresses of theselected core 505. The selected core 505, in some embodiments, can haveone or more IP addresses 630. In one embodiment, the packet engine 548can select one of the IP addresses 630 and replace the client IP addresswith the selected IP address 630. Upon modifying the tuple with theselected IP address 630, the first tuple is modified to comprise aselected IP address 630, the client port, the destination IP address andthe destination port.

In some embodiments, the packet engine 548 selects a port from amongstthe ports 632 of the selected core 505. In one embodiment, the packetengine 548 selects a port by iteratively applying the above-describedhash to each possible IP address 630 and port 632 combination. Thepacket engine 548 selects a port 630 which, when included in the firsttuple, identifies the selected core 505 when the above-described hash isapplied to the first tuple. For example, the packet engine 548 canselect an IP address 630 and then modify the first tuple with eachavailable port 632 of the selected core 505 until the output of the hashidentifies the selected core 505. In some embodiments, the packet engine548 modifies the tuple with the selected port. Modifying the tuple cancomprise inputting the selected port (Step 714) into the tuple orreplacing the client port with the selected port. Once the tuple hasbeen modified with the selected port, the tuple then comprises thefollowing values: the selected IP address; the destination IP address;the selected port; and the destination port.

The packet engine 548, in most embodiments, transmits the data packetwith the modified tuple to the client or server (Step 716). If the datapacket originated at a server, then in many embodiments, the packetengine 548 transmits the data packet to a client and vice versa. In someembodiments, the data packet transmits the data packet to a computingmachine or appliance corresponding to the destination IP address. Inother embodiments, the packet engine 548 transmits the data packet to anintermediary or proxy server or appliance prior to transmitting the datapacket to a destination computing machine or appliance.

Once the data packet is transmitted to a destination computing machineor appliance, the multi-core system 545 can receive another data packet(Step 704). In some embodiments, the method 700 can occur on a continualbasis so long as the multi-core system 545 receives and transmits datapackets and network traffic. While FIG. 7A illustrates a single instanceof the method 700 where each step occurs individually, in otherembodiments, multiple steps of the method 700 can occur simultaneously.For example, the packet engine 548 can receive a forwarded data packet(Step 710) at substantially the same time as the multi-core system 545receives a data packet from a client or a server (Step 704). In anotherexample, a packet engine 548A on a first core 505A receives a forwardeddata packet (Step 710) at substantially the same time as a packet engine548B on a second core 505B receives a forwarded data packet (Step 710).Therefore, any number of steps can occur at substantially the same time,including the same step.

Illustrated in FIG. 8 is an embodiment of a method 800 for distributingdata packets amongst cores 505 in a multi-core system 545. In oneembodiment, the multi-core system 545 receives a data packet (Step 802)and a flow distributor 550 or RSS module 560 identifies a tuple of thedata packet (Step 804). After identifying the tuple, the above-describedhash is applied to the identified tuple (Step 806) to generate aresultant value. The resultant value, in most embodiments, identifies acore 505 in the multi-core system 545. The RSS module 560 or the flowdistributor 550 transmits the data packet to the core 505 identified bythe resultant hash value (Step 808). In some embodiments, a packetengine 548 on the selected core 505 receives the data packet and selectsan IP address and port of the selected core 505 (Step 810). The packetengine 548 can then determine whether a hash of the selected IP address,the selected port and a portion of the tuple generates a value thatidentifies the selected core 505. When it is determined that the valuegenerated by the hash applied to the above-mentioned tuple identifiesthe selected core 505, the packet engine 548 modifies the tuple with theselected IP address and port (Step 814). Upon modifying the tuple, thepacket engine 548 or another module executing on the selected core 505forwards the modified data packet to a remote computing machine (Step816).

Further referring to FIG. 8, and in more detail, in one embodiment a NIC552 in the multi-core system 545 receives one or more data packetstransmitted to the multi-core system 545 over a network 104 (Step 802).In one embodiment, a flow distributor obtains data packets from the NIC552. In other embodiments, a RSS module 560, packet engine 548 or otherdistribution module or program drains or otherwise obtains data packetsfrom the NIC 552. The flow distributor can drain or obtain data packetsfrom a receive queue on the NIC 552.

In some embodiments, the data packets received are a client request,while in other embodiments the data packets received are a serverresponse. To ensure that a server response is handled by the same core505 that handled the client request, the packet engine 548 that executeson the first core 505, selects an IP address and a port number that willcause a server response to be distributed to the first core 505. This IPaddress and port number are selected so that, when they are combinedwith the destination IP address and the destination port number, theresulting tuple identifies the first core 505. This resulting tuple,i.e. a second tuple, identifies the first core 505 in that when theabove-described hash function is applied to the second tuple, the hashresult identifies the first core 505. When the server generates aresponse, the response includes the selected IP address, the selectedport number, the destination IP address and the destination port number.Thus, when the flow distributor 550 applies the above-described hash tothe tuple within the server response, the hash result will identify thefirst core 505 and the server response is forwarded or allocated to thefirst core 505.

Once the flow distributor 550 receives the data packets, the flowdistributor or a distribution module can identify a tuple of the datapacket (Step 804). The tuple, in some embodiments, can comprise anycombination of the following values: a client IP address; a destinationIP address; a client port; a destination port; or any other IP address,port or other source or destination identifying value. The client IPaddress, in some embodiments, can be a source IP address. Similarly, theclient port, in some embodiments, can be a source port. Identifying atuple of the data packet can, in some embodiments, comprise generatingthe tuple by concatenating any of the above-mentioned values to create astring. The tuple, in some embodiments, is a string or array of values.

A hash or hash value is, in some embodiments, calculated by applying theabove-described hash to the identified tuple (Step 806). The hash valuecan be referred to by any of the following designations: hash; hashvalue; result value; result; or value. The hash can be applied by theflow distributor 550 or by any other distribution module executingwithin the multi-core system 545.

After applying the hash, a determination can be made as to whether theresultant value identifies a core 505 in the multi-core system 545. Whenthe hash result identifies a particular core 505, the data packet isforwarded to the identified core 505 by the flow distributor 550 or byany other flow distribution module (Step 808). In one embodiment, theflow distributor 550 forwards the data packet to a packet engine 548executing on the identified core 505. The identified core 505 can bereferred to as a first core 505. In some embodiments, the hash resultmay not identify a core 505 within the multi-core system 545. In theseembodiments, the data packet can be forwarded to a default core 505 inthe multi-core system 545. In still other embodiments, the data packetmay not have an associated tuple. In those embodiments, the data packetcan be forwarded to a default core 505 in the multi-core system 545.

Upon forwarding the data packet to the identified core 505, a packetengine 548 or other module or engine executing on the identified core505 can receive the forwarded data packet. In some embodiments, acommunication module executing on the identified core 505 receives thedata packet and forwards the data packet to a packet engine 548 on theidentified core 505. Once the packet engine 548 receives the forwardedpacket, the packet engine 548 can select an IP address of the core 505and a port of the core (Step 810). This IP address, in some embodiments,can be an IP address of the multi-core system 545 or an IP address ofthe appliance 200. In other embodiments, the IP address can be an IPaddress of the core 505. The core 505 can have one or more IP addresses,therefore in some embodiments the packet engine 548 can select an IPaddress based on a determination as to whether the IP address combinedwith a selected port and a portion of the first tuple identifies theidentified core 505.

Selecting a port of the core 505 can include searching through portsassociated with the selected core 505 to identify a port that whenincluded in the first tuple, identifies the selected core 505. In someembodiments, the packet engine 548 can iterate through each IP addressof the core 505 and each port of the core 505 to identify an IPaddress/port combination that identifies the selected core 505. Forexample, the selected core 505 can be a first core 505 having a tuplecomprising a client IP address, a client port, a destination IP addressand a destination port. The packet engine 548 can modify the tuple toinclude a selected IP address, a selected port, the destination IPaddress and the destination port. Before permanently modifying the datapacket, the packet engine 548 first applies the above-described hash tothe modified tuple (Step 812). If the resultant hash value identifiesthe first core 505, then the packet engine 548 permanently modifies thedata packet to replace or change the client IP address to the selectedIP address, and replace or change the client port to the selected port.If the resultant hash value does not identify the first core 505, thenthe packet engine 548 changes either or both of the selected IP addressand the selected port, and applies the hash again.

In some embodiments, selecting either the port number or the IP addresscan include selecting an IP address from amongst one or more IPaddresses of the first core 505 or the packet engine 548 executing onthe first core 505, and selecting a port number from a port tableassociated with the first core 505 or from amongst one or more portnumbers associated with the first core 505 or the packet engine 548. Thepacket engine 548 can select an IP address and a first port number. Upondetermining that the first port number is not available, the packetengine 548 can select a second port number and determine that the secondport number is available. Upon determining that the second port number,the packet engine 548 can then apply the above-described hash to afourth tuple comprising the selected IP address, the second port number,the destination IP address and the destination core. Upon determiningthat the resulting hash value identifies the first core 505, the packetengine 548 modifies the client request to include the selected IPaddress, the second port number, the destination IP address and thedestination core. In other embodiments, the packet engine 548 candetermine that the first port is not available, can select a second IPaddress from amongst the available IP addresses, and can select a secondport number from the plurality of ports associated with the first core505. The packet engine 548 can then apply the above-described hash to afifth tuple comprising the second IP address, the second port number,the destination IP address and the destination port. Upon determiningthat the resulting hash identifies the first core 505, the packet engine548 can update the client request with the fifth tuple so that theclient request identifies the second IP address and the second portnumber.

After applying the above-described hash (Step 812) to verify that theselected IP address and the selected port, when combined with thedestination IP address and the destination port, identify the selectedcore 505, the packet engine can then modify the data packet so that thetuple comprises: the selected IP address; the destination IP address;the selected port; and the destination port (Step 814). In thisembodiment, the client IP address and the client port are no longerincluded within the tuple. Rather, these values have been replaced bythe selected IP address and the selected port.

The packet engine 548, in many embodiments, transmits the updated datapacket and tuple to a remote computing device (Step 816) after modifyingthe data packet and tuple. In some embodiments, the remote computingdevice can be a client, a server or another computing machine orappliance located remote from the multi-core system 545. In otherembodiments, the packet engine 548 can transmit the modified data packetto an intermediary device which forwards the data packet to adestination location. The destination location, in some embodiments, isidentified by the destination IP address and/or the destination port.

The method 800, in some embodiments, can further include the flowdistributor 550 receiving a response to the client request allocated tothe first core 505. The response can be generated by a server and cancomprise a tuple, (i.e. a second tuple or third tuple,) comprising theselected IP address, the selected port number, the destination IPaddress and the destination port. The flow distributor 550 applies ahash to the tuple of the response and the resulting hash valueidentifies the first core 505. Subsequent to this determination, theflow distributor allocates or distributes the server response to thefirst core 505 or to a packet engine 548 executing on the first core505.

In still other embodiments, the method 800 can further include a packetengine 548 executing on the first core 505, the packet engine 548updating a port allocation table associated with the first core 505and/or the packet engine 548. This port allocation table can be updatedby the packet engine 548 with an entry or with information indicatingthat the selected port number included in the second tuple of the serverresponse and the modified client request has been assigned to a datapacket. Thus, any subsequent data packets or requests handled by thefirst packet engine 548 on the first core 505 cannot receive theselected port number because the port allocation table identifies thatport number as unavailable.

One example of the method 800, as applied to a client request andcorresponding server response includes receiving, by a flow distributor550, a client request generated by a client communicating with themulti-core system 545. The flow distributor 550 identifies a first tupleof the client request, the first tuple comprising a client IP address, adestination IP address, a client port and a destination port. Afteridentifying the first tuple, the flow distributor 550 applies theabove-described hash function to the first tuple to generate a hashresult that identifies a first core 505 in the multi-core system 545.The flow distributor 550 then forwards the client request to the firstcore 505 where it is received by a first packet engine executing on thefirst core 505. The first packet engine 548 receives the client request548 and selects an IP address of the first core 505 or first packetengine 548 and a port number of the first core 505 or the first packetengine 548. The IP address and port are selected so that a hash of asecond tuple comprising the selected IP address, the destination IPaddress, the selected port and the destination port will generate a hashresult that identifies the first core 505. This is done so that anyresponse to the client request will be distributed to the first core 505rather than to another core 505 in the multi-core system 545. Ensuringthat the same core 505 processes both the request and response reducesthe need to generate unnecessary data copies of data packets handled bythe multi-core system 545 and ensures symmetric request/responseprocessing. Once an IP address and port number is selected by the firstpacket engine 548, the first packet engine 548 transmits the clientrequest to a server. The flow distributor 550 then receives a serverresponse to the client request, where the server response has a secondtuple comprising the selected IP address, the selected port number, thedestination IP address and the destination port. The flow distributor550 applies the above-described hash to the second tuple and the hashresult identifies the first core 505. Accordingly, the flow distributor550 forwards the server response to the first packet engine 548 on thefirst core 505 for processing.

Illustrated in FIG. 9 is an embodiment of a method 900 for distributingnetwork traffic amongst cores 505 in a multi-core system 545. The method900 described in FIG. 9 illustrates how a packet engine 548 on a core505 handles a received data packet. The packet engine 548 receives anallocated data packet (Step 902) and selects an IP address of the core505 on which the packet engine 548 executes (Step 904). The packetengine 548 also selects a port number from a plurality of port numbersallocated or associated with the packet engine 548 or the core 505 (Step906). Once an IP address and port number are selected, the packet engine548 then determines whether a hash of the selected IP address and theselected port number together with a destination IP address anddestination port number, will identify the current core 505. Inparticular, the packet engine 548 determines whether the selected portnumber will identify the current core 505 (Step 908). When it isdetermined that the selected port number will not identify the currentcore 505, the packet engine 548 selects the next port number fromamongst the port numbers associated with the core 505 (Step 906). Whenit is determined that the selected port number will identify the currentcore 505, the packet engine 548 next determines whether the selectedport number is open or otherwise available (Step 910). When it isdetermined that the selected port number is not open, the packet engine548 selects the next port number from amongst the port numbersassociated with the core 505 (Step 906). When it is determined that theselected port number is open or otherwise available, the packet engine548 modifies the data packet with the selected IP address and theselected port number (Step 912) and forwards the data packet and itsmodified tuple to a remote computing machine (Step 914).

Further referring to FIG. 9, and in more detail, in one embodiment themethod 900 can be carried out by a packet engine 548 executing on a core505. In another embodiment, the method 900 can be carried out by a flowdistributor 550 or instance of a flow distributor executing on the core505. In still other embodiments, the method 900 can be carried out byany flow distribution module or agent that may execute on the core 505.While FIG. 9 contemplates processing a data packet that can be modifiedin part on a particular core 505, modification of the data packet can behandled, in some embodiments, by a control core in the multi-core system545.

The packet engine 548 carrying out the steps of the method 900 describedin FIG. 9 can execute on a particular core 505. The core 505, in mostembodiments, is selected ahead of time by the method 800 illustrated inFIG. 8. Therefore in most instances, the data packet received by thepacket engine 548 has been allocated to the core 505 based on theapplication of an above-described hash to a tuple of the data packet.This tuple, in most cases, comprises at least a client IP address, adestination IP address, a client port and a destination port. In someembodiments, the tuple can be any of the above described tuples and cancomprise any number of source or destination identifying values. Instill other embodiments, the client IP address can be a source IPaddress identifying the machine from which the data packet originated.Similarly, the client port can be a source port.

In one embodiment, a packet engine 548 executing on a particular core505 in the multi-core system 545, receives data packets allocated tothat particular core 505 (Step 902).

The packet engine 548 can directly receive data packets, or in someembodiments, a communication module executing on the core 505 canreceive and transmit data packets. Receiving data packets, in someembodiments, can include receiving requests and responses such as clientrequests and server responses. In other embodiments, a virtual NIC (NotShown) executing on the core 505 can receive and transmit data packets.Receiving data packets, in some embodiments, can further comprisedraining data packets from a logical receive queue on the core 505. Alogical receive queue can store data packets transmitted to a core 505.The packet engine 548 can access data packets in the logical receivequeue by draining or otherwise obtaining the data packets from thereceive queue according to a first-in-first-out method of access.Another possible method of access can be first-in-last-out.

When a packet engine 548 obtains a data packet, the packet engine 548can in some embodiments determine whether the data packet can bemodified. The packet engine 548, after determining what portions of thedata packet can be modified, can modify the data packet. In someembodiments, the multi-core system 545 can be configured to instructpacket engines 548 executing within the multi-core system 545 to modifyonly certain portions of the data packet.

In some embodiments, the packet engine 548 can select an IP address ofthe core 505 from amongst one or more IP addresses associated with thecore 505 (Step 904). The core 505 can have multiple IP addresses, and insome embodiments can have a range of IP addresses. In other embodiments,the core 505 can have a single IP address. While in some embodiments thepacket engine 548 selects an IP address of the core 505, in otherembodiments the packet engine 548 can select an IP address of themulti-core system 545 or an appliance 200 in the multi-core system 545.

Once the IP address is selected, the packet engine 548 can then select aport number from amongst a plurality of port numbers of the core 505(Step 906). The core 505 can have one or more port numbers, and in someembodiments can store in a port allocation table a listing of each ofthe port numbers 505 of a multi-core system 545. Selecting a port cancomprise cycling through the entries of a port allocation table listingeach of the ports of a core 505 and selecting a port number. The portnumbers can be cycled through numerically based on port number or basedon the order in which the ports are listed in the port allocation table.In other embodiments, the packet engine 548 can select a port by cyclingthrough a range of numbers or values corresponding to possible portnumbers on the core 505.

In some embodiments, the packet engine 548 can select a first portnumber (step 906) and then determine whether that port is the correctport (Step 908) and whether that port is available or open (step 910).If the selected first port is either not the correct port number or notavailable or open, the packet engine 548 can select the a next portnumber, i.e. a second port number of the core 505, and again determinewhether that port number is the correct port number (Step 908) andwhether that port number is available or open (Step 910). In mostembodiments, the packet engine 548 cycles through all possible portnumbers until the packet engine 548 identifies a port number that isboth the correct port number and an open port number.

Once the packet engine 548 selects a port number, the packet enginefirst determines whether the selected port number is the correct portnumber by determining whether the selected port number will cause aresponse packet to return to the selected core (Step 908). Thisdetermination can be made by applying the above-described hash to atuple comprised of a concatenation of the following values: the selectedIP address; the destination address; the selected port number; and thedestination port number. Applying the above-described hash to this tuplegenerates a resultant hash value that either identifies or does notidentify the core 505 on which the packet engine 548 is currentlyexecuting. Concatenating the tuple values to generate the tuple can becarried out by the packet engine 548 or by an instance of a flowdistributor 550 executing on the core 505. Similarly, applying the hashto the tuple can be carried out by the packet engine 548 or by aninstance of a flow distributor. When the resultant hash value identifiesthe current or selected core 505, the selected port number is thecorrect port number because it will cause a response packet to return tothe current core 505. When the resultant hash value does not identifythe current or selected core 505, the selected port number is not thecorrect port number because it will not cause a response packet toreturn to the current core 505. In this situation, the packet engine 548will select another port number (Step 906) and reiterate the process ofdetermining whether the port number is the correct port number (Step910).

When it is determined that a selected port number is the correct portnumber (Step 908), a determination is then made as to whether the portnumber is available or open (Step 912). In most embodiments, a portnumber is open or available when any of the following is true: the portnumber is not being used; or the port number is available for use. Incontrast, a port number is not open or available when any of thefollowing is true: the port number has been assigned to anothertransaction, service or data packet; or the port number has been closedeither by a network administrator or by the multi-core system 545.Whether a port number is available or open, is a characteristic that inmany embodiments is tracked by a port number allocation table. The portallocation table can any of the above-mentioned port allocation tablesand can be stored in any of the above-mentioned locations that a porttable can be stored. In some embodiments, after the packet engine 548determines that the port number is the correct port number, the packetengine 548 can determine that the port number is available by querying aport allocation table for the details, attributes or characteristics ofa particular port number. When the response indicates both that the portnumber is open and that the port number has not been assigned to anyother data packet, transaction, or server, then the packet engine 548modifies the tuple with the selected IP address and the selected portnumber. However, when the response indicates that the port number iseither not available or not open, the packet engine 548 selects anotherport number (Step 906) and repeats the process of determining whetherthe port number is the correct port number (Step 908) and whether theport number is open and available (Step 910).

When a correct, open and available port number is selected by the packetengine 548, the packet engine 548 then updates the data packet andtherefore the tuple of the data packet to include the selected IPaddress and the selected port number (Step 912). Modifying or updatingthe tuple can comprise making any modification necessary to cause thetuple to comprise: the selected IP address; the destination IP address;the selected port number; and the destination port number. Thus, theclient IP address and the client port number information can be replacedby the selected IP address and the selected port number.

After modifying the data packet, the packet engine 548 can transmit themodified data packet to a remote computing machine (Step 914).Transmitting the modified data packet to a remote computing machine cancomprise transmitting the modified data packet to a client, server,appliance, or computing machine identified by the destination IP addressand/or the destination port number. In some embodiments, the modifieddata packet is transmitted to a proxy server or appliance before thedata packet is transmitted to its destination computing machine orappliance. In other embodiments, the modified data packet is stored in amemory element within the multi-core system 545 before the data packetis transmitted to its destination computing machine or appliance. Thememory element, in some embodiments, can be a global cache or othermemory element shared by all cores and devices in the multi-core system545. In other embodiments, the memory element can be a cache or otherstorage repository accessible by the current core 505.

3. System and Method for Retaining Source IP in a Load BalancingMulti-Core Environment and Proxying the Source Port

While FIGS. 7A, 8 and 9 describe methods where the client IP address andthe client port are modified or replaced by an IP address and portselected by a packet engine 548 on a particular core 505, FIGS. 7B, 12Aand 12B describe a system where the client IP address is maintained. Theclient port or source port, however, can be a proxy port that isselected by a packet engine and inserted into a tuple of a data packetin lieu of the client port. In some systems, the owner of a server farmor the administrator of a network within which the multi-core system 545executes can desire that each data packet retain at least its originalsource IP address. An administrator may want to do this for any numberof reasons, some of those reasons can include for security purposes, formarketing purposes, to track network access, to restrict network access,or for any other reason. By permitting each data packet to retain itssource IP address, each data packet can be tracked and controlled. Forexample, knowing the source of a data packet can permit the system toprevent particular IP addresses or domains from accessing a network.Similarly, knowing the source of a data packet can permit the system totrack the geographic location of users accessing the network or domain.In most cases, knowing the source IP address allows a system to identifythe location of where a packet originates and to further control whetheror not a particular data packet is processed by a system.

Given that only the client port number can be modified, the number ofselected port numbers can that when combined with a maintained client IPaddress identify a current core can, in some embodiments, become scarce.Therefore each core 505 can be associated with multiple port allocationtables where each port allocation table stores a list of available portnumbers. Permitting each core 505 to be associated with one or more portallocation tables in addition to one or more port numbers adds anotherlayer of uniqueness in that each request can now be associated with aport number from a particular port allocation table. This additionallayer of uniqueness can overcome port number scarcity caused bymaintaining the client IP address.

Illustrated in FIG. 7B is a flow diagram depicting one embodiment of amethod 780 for using the above-discussed hash to distribute networktraffic amongst one or more cores 505 in a multi-core system 545. Thismethod 780 is similar to the method 700 illustrated in FIG. 7A. Howeverin the method 780 illustrated in FIG. 7B, the packet engine 548maintains the client IP address. Like the method 700 illustrated in FIG.7A, a flow distributor 550 or RSS module 560 receives data packets froma client, server or other computing machine (Step 782), and calculates ahash value by applying the hash to a first tuple of the received datapacket (Step 784). The first tuple can comprise a client IP address, adestination IP address, a client port, and a destination port. Applyingthe hash to the first tuple can, in some embodiments, result in a valuesometimes referred to as the hash. A core 505 in the multi-core system545 is selected based on the hash result value (Step 786) and thereceived data packet is forwarded to the selected core (Step 788). Atthis point the first tuple still comprises the following values: clientIP address; destination IP address; client port; and destination port. Apacket engine 548 on the selected core receives the data packet andmaintains the client IP address (Step 709), but updates the first tuplewith a selected port (Step 792). The first tuple, at this point,comprises the following values: the client IP address; the destinationaddress; the selected port; and the destination port. The data packetand its modified tuple are then transmitted to a server, client or othercomputing machine (Step 794). Any responses to this data packetgenerated by the server, client or other computing machine are forwardedto the multi-core system 545 and received by the multi-core system 545(Step 782). At this point, the method 700 repeats itself.

Further referring to FIG. 7B, and in more detail, in one embodiment themethod 780 illustrated in FIG. 7B differs from the method 700illustrated in FIG. 7A in that the method 780 illustrated in FIG. 7Bmaintains the client or source IP address. Thus, the additional stepsare substantially the same as the steps described in the method 700illustrated in FIG. 7A. For example, like the previously describedmethod 700, the multi-core system 545 can receive data packets from aclient, server or other computing machine (Step 782). Step 782 can, insome embodiments, be any of the embodiments of Step 704 described inFIG. 7A. Like the above-described method 700, a hash is applied to afirst tuple of the data packet (Step 784), and a core is selected basedon the result of the hash (Step 786). Step 784 can be any of theembodiments of Step 706 described in FIG. 7A, while Step 786 can be anyof the embodiments of Step 708 described in FIG. 7A. Once a core 505 isselected, the data packet can be forwarded to the selected core 505(Step 788). Step 788 can be any of the embodiments of Step 710. Afterthe tuple associated with the data packet is modified, the modified datapacket is then transmitted to a server, client or other computingmachine (Step 794). Step 794 can be any of the embodiments of Step 716.

In some embodiments, once the data packet is forwarded to a selectedcore 505 (Step 788), a packet engine 548 or other engine or moduleexecuting on the selected core 505, can receive the packet and determinewhether the packet can be modified. Determining whether a data packetcan be modified can include making any of the following determinations:whether a portion of the data packet can be modified; whether a tuple ofthe data packet can be modified; whether any portion of a tuple of thedata packet can be modified; what portions of the data packet and/ortuple can be modified; and any other determinations that may impactwhether the packet engine 548 can modify the data packet or a tuple ofthe data packet. In one embodiment, the packet engine 548 determinesthat a portion of the data packet can be modified, and in particularthat a portion of a tuple of the data packet can be modified. Thisdetermination can further include a determination that the client IPaddress, or source IP address, of the data packet should be maintainedand therefore cannot be modified. Based on this determination, thepacket engine 548 can adjust packet processing according to thedetermination. In some embodiments, the determination can be made byanalyzing the data packet, a header of the data packet or any otherattribute of the data packet. In other embodiments, the multi-coresystem 545 can be configured to maintain the client IP address andfurther proxy the client port. In these embodiments, a determination asto whether the data packet or a tuple of the data packet can be modifiedis not made because the operation of the system 545 is configuredaccordingly.

When either a determination is made that the client IP address should bemaintained or when the system 545 dictates that the client IP addressshould be maintained, the packet engine 548 maintains the client IPaddress (Step 790) rather than modifying the tuple to include an IPaddress of the core 505 or system 545. After this step, the tuplecomprises the following values: the client IP address; the destinationIP address; the client port; and the destination port.

Maintaining the client IP address can cause any response to the datapacket to be routed to a different core than the selected core 505.Therefore, the packet engine 548 should identify and select a port 632from amongst the ports 632 of the selected core 505, that when includedin the tuple in lieu of the client port, causes a hash of the tuple toidentify the selected core 505. Thus, the packet engine 548 iteratesthrough each of the ports 632 of the core 505 to identify such a port632 and selects the port 632. After selecting the port 632, the packetengine 548 updates the tuple of the data packet to include the selectedport 632 (Step 792). After this step, the tuple comprises the followingvalues: the client IP address; the destination IP address; the selectedport; and the destination port.

The updated data packet and tuple are then transmitted to a server,client or computing machine (Step 794). The data packet, whentransmitted, comprises a tuple comprising the following values: theclient IP address; the destination IP address; the selected port; andthe destination port.

Illustrated in FIG. 12A is one embodiment of a method 1200 fordistributing packets across a multi-core system 545. In the method, aflow distributor 550 or RSS module 560 receives a data packet (Step1202) and identifies a tuple of the data packet (Step 1204). Afteridentifying the tuple, a hash is applied to the tuple to generate aresult (Step 1206) and the data packet is transmitted to a coreidentified by the hash result (Step 1208). The data packet, in someembodiments, can be received by a packet engine 548 on the core. Thepacket engine 548 can maintain a client IP address included in the tuple(Step 1210), but can select a port from amongst the ports of the core(Step 1212) and can modify the tuple with the determined port (Step1214). Once the tuple is modified, the data packet and the modifiedtuple is transmitted to a remote computing machine (Step 1216).

Further referring to FIG. 12A, and in more detail, in one embodiment themethod 1200 is substantially the same as the method illustrated in FIG.8. Therefore Step 1202 can be any embodiment of Step 802 illustrated inFIG. 8, similarly Step 1204 can be any embodiment of Step 804illustrated in FIG. 8. Step 1206 can be any embodiment of Step 806illustrated in FIG. 8, Step 1208 can be any embodiment of Step 808illustrated in FIG. 8, and Step 1216 can be any embodiment of Step 816illustrated in FIG. 8. In some embodiments, the method 1200 illustratedin FIG. 12A differs from the method 800 illustrated in FIG. 8 in thatthe method 1200 illustrated in FIG. 12A maintains the client IP address.

The packet engine 548 carrying out the steps of the method 1200described in FIG. 12A can execute on a particular core 505. Therefore inmost instances, the data packet received by the packet engine 548 hasbeen allocated to the core 505 based on the application of anabove-described hash to a tuple of the data packet. This tuple, in mostcases, comprises at least a client IP address, a destination IP address,a client port and a destination port. In some embodiments, the tuple canbe any of the above described tuples and can comprise any number ofsource or destination identifying values. In still other embodiments,the client IP address can be a source IP address identifying the machinefrom which the data packet originated. Similarly, the client port can bea source port.

In one embodiment, a packet engine 548 executing on a particular core505 (i.e. a first core 505) in the multi-core system 545, receives datapacket allocated to that particular core 505 (Step 1208). The packetengine 548 can directly receive data packets, or in some embodiments, acommunication module executing on the core 505 can receive and transmitdata packets. In other embodiments, a virtual NIC (Not Shown) executingon the core 505 can receive and transmit data packets. Receiving datapackets, in some embodiments, can further comprise draining data packetsfrom a logical receive queue on the core 505. A logical receive queuecan store data packets transmitted to a core 505. The packet engine 548can access data packets in the logical receive queue by draining orotherwise obtaining the data packets from the receive queue according toa first-in-first-out method of access. Another possible method of accesscan be first-in-last-out. The packet engine 548, in some embodiments,can receive a client request or a server response.

When a packet engine 548 obtains a data packet, the packet engine 548can in some embodiments determine whether the data packet can bemodified. The packet engine 548, after determining what portions of thedata packet can be modified, can modify the data packet. In someembodiments, the multi-core system 545 can be configured to instructpacket engines 548 executing within the multi-core system 545 to modifyonly certain portions of the data packet.

In some embodiments, the packet engine 548 can determine that the datapacket cannot be modified. In other embodiments, the multi-core system545 can be configured such that the data packet is not modified, butrather each element of the tuple of the data packet is maintained exceptfor the client port. Thus, when the packet engine 548 receives the datapacket, the packet engine 548 maintains the client IP address, i.e. thesource IP address (Step 1210).

In some embodiments, the packet engine 548 determines, prior tocomputing a second hash on the client IP address, the destination IPaddress, a selected port number and the destination port, to proxy theclient port and maintain the client IP address. Determining to proxy theclient port can, in some embodiments include determining to select aport from amongst the ports of the first core 505 and replace the clientport with that selected port.

In one embodiment, the packet engine 548 selects a port from amongst theports of the core 505 (Step 1212). The selected port, in someembodiments, is a proxy port that can be included in the first tuple inlieu of the client port. This proxy port can be determined so that ahash of the modified first tuple will identify the current core 505.This determination can be made by applying the above-described hash to asecond tuple comprising the client IP address, the destination IPaddress, a selected port number and the destination port. When theresult of this hash identifies the first core 505, then it can bedetermined that the selected port will cause a response to the datapacket to be allocated to the current core 505. The determination canalso include determining whether the port is available. When a port isnot available or otherwise is assigned to a data packet, the packetengine 548 can select a second port and determine whether that portnumber will cause a response packet to be routed or distributed to thefirst core 505. Once the port is determined, the first tuple is modifiedwith the identified port (Step 1214), and the modified data packet andtuple are forwarded to a remote computing machine (Step 1216). The datapacket, when transmitted, retains a tuple comprising the followingelements: client IP address; destination IP address; selected port; anddestination port.

Selecting a port, in some embodiments, further comprises selecting aport number from a port allocation table associated with the first core505. The port allocation table can be one of multiple port allocationtables associated with the first core 505, and can be located at a proxyIP address of the first core 505. In one embodiment, the packet engine548 selects a first port number from amongst multiple port numbers, anddetermines that a hash of a second tuple comprising the client IPaddress, the destination IP address, the first port number and thedestination port number, does not identify the first core 505. Uponmaking this determination, the packet engine 548 selects a second portnumber from the same port allocation table and determines that a thirdtuple comprising the client IP address, the destination IP address, thesecond port number and the destination port number, does not identifythe first core 505. The packet engine 548, in some embodiments, selectsthe second port number based on a determination that the first portnumber is not available. In still another embodiment, the packet engine548 selects a first port number from amongst multiple port numbers in aport allocation table selected based on the result of applying theabove-described hash to a tuple comprising the client IP address and thedestination IP address.

Transmitting the modified data packet to a remote computing machine cancomprise transmitting the modified data packet to a client, server,appliance, or computing machine identified by the destination IP addressand/or the destination port. In some embodiments, the modified datapacket is transmitted to a proxy server or appliance before the datapacket is transmitted to its destination computing machine or appliance.In other embodiments, the modified data packet is stored in a memoryelement within the multi-core system 545 before the data packet istransmitted to its destination computing machine or appliance. Thememory element, in some embodiments, can be a global cache or othermemory element shared by all cores and devices in the multi-core system545. In other embodiments, the memory element can be a cache or otherstorage repository accessible by the current core 505.

Illustrated in FIG. 12B is one embodiment of a method 1250 for selectinga port from a port allocation table of the selected core 505. A packetengine 548 on the selected core 505 calculates a hash of the client IPaddress and the destination IP address (Step 1252), the hash identifyinga port allocation table on the selected core 505 (Step 1254). Once aport allocation table is selected, a port in the port allocation tableis selected (Step 1256) and a determination is made as to whether theport is open (Step 1258). A tuple of the data packet is then modifiedwith the determined port (Step 1260) and the modified data packet andtuple are forwarded to a remote computing machine (Step 1262).

Further referring to FIG. 12B, and in more detail, in one embodiment apacket engine 548 executing on the selected core 505 calculates a hashvalue of the client IP address and the destination IP address (Step1252). Calculating the hash value can comprise concatenating the clientIP address and the destination IP address to create a string or two itemtuple. The packet engine 548 then applies the above-described hashfunction to the two tuple to generate a resultant value or hash value.This hash value, in many embodiments, identifies a port allocation tableon the selected core 505 (Step 1254). There may, in some embodiments, bemultiple port allocation tables associated with a particular core 505.Determining which port allocation table from which to select a port cancomprise generating the hash value and using the hash value to select acorresponding port allocation table.

In most embodiments, once the packet engine 548 selects a portallocation table, the packet engine 548 can then select a port from theport allocation table (Step 1256). When a port is selected adetermination must be made as to whether the port is both the correctport and an open port (Step 1258). This determination can be made viathe method 900 illustrated in FIG. 9. When it is determined that theport is the incorrect port and/or closed and unavailable, the packetengine 548 can select a different port in the selected port allocationtable. Once the new port is selected, a determination must be made as towhether the port is both the correct port and open. In some embodiments,there are no ports in the port allocation table that are both thecorrect port and an available port. In these embodiments, a differentport allocation table can be chosen. A port is then selected from thenewly chosen port allocation table and a new determination is made as towhether the selected port is both the correct port and an availableport.

Once a port is selected that is both the correct port and an open port,the tuple of the data packet can be modified with the selected port(Step 1260). Upon modifying the tuple with the selected port, themodified data packet can be transmitted to a remote computing machine(Step 1262).

Transmitting the modified data packet to a remote computing machine cancomprise transmitting the modified data packet to a client, server,appliance, or computing machine identified by the destination IP addressand/or the destination port. In some embodiments, the modified datapacket is transmitted to a proxy server or appliance before the datapacket is transmitted to its destination computing machine or appliance.In other embodiments, the modified data packet is stored in a memoryelement within the multi-core system 545 before the data packet istransmitted to its destination computing machine or appliance. Thememory element, in some embodiments, can be a global cache or othermemory element shared by all cores and devices in the multi-core system545. In other embodiments, the memory element can be a cache or otherstorage repository accessible by the current core 505.

4. System and Method for Retaining Source IP and Source Port in a LoadBalancing Multi-Core Environment

While FIGS. 7A, 8 and 9 describe methods where the client IP address andthe client port are modified or replaced by an IP address and portselected by a packet engine 548 on a particular core 505, and FIGS. 7B,12A and 12B describe a system where the client IP address is maintained;FIGS. 7C, 10A and 10B describe a system where the client IP address andthe client port are maintained. In some systems, the owner of a serverfarm or the administrator of a network within which the multi-coresystem 545 executes can desire that each data packet retain its originalsource IP address and source port. An administrator may want to do thisfor any number of reasons, some of those reasons can include forsecurity purposes, for marketing purposes, to track network access, torestrict network access, or for any other reason. By permitting eachdata packet to retain its source IP address or source port, each datapacket can be tracked and controlled. For example, knowing the source ofa data packet can permit the system to prevent particular IP addressesor domains from accessing a network. Similarly, knowing the source of adata packet can permit the system to track the geographic location ofusers accessing the network or domain. In most cases, knowing the sourceIP address and source port allows a system to identify the location ofwhere a packet originates and to further control whether or not aparticular data packet is processed by a system.

Illustrated in FIG. 7C is a flow diagram depicting one embodiment of amethod 750 for using the above-discussed hash to distribute networktraffic amongst one or more cores 505 in a multi-core system 545. Thismethod 750 is similar to the method 700 illustrated in FIG. 7A. Howeverin the method 750 illustrated in FIG. 7C, the packet engine 548maintains both the client IP address and the client port. Like themethod 700 illustrated in FIG. 7A, a flow distributor 550 or RSS module560 receives data packets from a client, server or other computingmachine (Step 766), and calculates a hash value by applying the hash toa first tuple of the received data packet (Step 756). The first tuplecan comprise a client IP address, a destination IP address, a clientport, and a destination port. Applying the hash to the first tuple can,in some embodiments, result in a value sometimes referred to as thehash. A first core 505A in the multi-core system 545 is selected basedon the hash result value (Step 758) and the received data packet isforwarded to the selected first core 505A (Step 760). At this point thefirst tuple still comprises the following values: client IP address;destination IP address; client port; and destination port. Once theselected core receives the forwarded data packet, the selected firstcore 505A determines whether that core is the correct core (Step 772).When a determination is made that the selected core 505A is the correctcore, then the method continues to Step 762. However, when adetermination is made that the selected core 505A is not the correctcore, the data packet is forwarded to the correct core (Step 774) beforeproceeding to Step 762. The packet engine 548 on either the first core505A, or on a correct core different from the first core 505A, maintainsthe client IP address and the client port (Step 762) after which thedata packet is transmitted to the server, client or other computingmachine (Step 764). Any responses to this data packet generated by theserver, client or other computing machine are forwarded to themulti-core system 545 and received by the multi-core system 545 (Step766). At this point, the method 750 repeats itself.

Further referring to FIG. 7C, and in more detail, in one embodiment themethod 750 illustrated in FIG. 7C differs from the method 700illustrated in FIG. 7A in that the method 750 illustrated in FIG. 7Cmaintains the client IP address and the client port. Thus, theadditional steps are substantially the same as the steps described inthe method 700 illustrated in FIG. 7A. For example, like the previouslydescribed method 700, the multi-core system 545 can receive data packetsfrom a client, server or other computing machine (Step 766). Step 766can, in some embodiments, be any of the embodiments of Step 704described in FIG. 7A. Like the above-described method 700, a hash isapplied to a first tuple of the data packet (Step 756), and a core isselected based on the result of the hash (Step 758). Step 756 can be anyof the embodiments of Step 706 described in FIG. 7A, while Step 758 canbe any of the embodiments of Step 708 described in FIG. A. Once a core505 is selected, the data packet can be forwarded to the selected core505 (Step 760). Step 760 can be any of the embodiments of Step 710.After the tuple associated with the data packet is modified, themodified data packet is then transmitted to a server, client or othercomputing machine (Step 764). Step 764 can be any of the embodiments ofStep 716.

In one embodiment, when the packet engine 548 on the selected core 505receives the forwarded data packet (Step 760), the packet engine 548determines whether the packet was previously handled by the currentcore. If the current core is not the correct core (Step 772), then thedata packet is forwarded to the correct core (Step 774). The correctcore can be determined by applying the above-described hash to a tupleof the data packet. Forwarding or otherwise transmitting the data packetto the correct core can be done via a core-to-core messaging systemand/or by copying the data packet into a global cache accessible by boththe current core and the correct core.

When the data packet is forwarded to the correct core, the first tuplecomprises the following values: a client IP address; a destination IPaddress; a client port; and a destination port. In embodiments where thecurrent core is the correct core, the current core maintains the clientIP address and the client port (Step 762). Similarly, when the correctcore receives the data packet, the correct core maintains the client IPaddress and the client port (Step 762). By maintaining the client IPaddress and the client port, the tuple continues to comprise thefollowing values: a client IP address; a destination IP address; aclient port; and a destination port. Once the client IP address and theclient port are maintained, the data packet is transmitted to theserver, client or other computing device or appliance.

Illustrated in FIG. 10A is a method 1000 for allocating a data packet toa particular core 505 in a multi-core system 545. The method 1000includes receiving a data packet (Step 1002), identifying a tuple of thedata packet (Step 1004) and applying a hash to the tuple (Step 1006).The data packet is then forwarded to a core 505 in the multi-core system545 (Step 1008), where the core 505 is identified by a value resultingfrom the application of any of the above-mentioned hashes to a tuple ofthe data packet. A packet engine 548 executing on the selected core 505maintains both the client IP address and the client port of the tuple(Step 1010), and forwards the data packet and un-modified tuple to aremote computing machine (Step 1012).

Further referring to FIG. 10A, and in more detail, in one embodiment themethod 1000 is substantially the same as the method illustrated in FIG.8. Therefore Step 1002 can be any embodiment of Step 802 illustrated inFIG. 8, similarly Step 1004 can be any embodiment of Step 804illustrated in FIG. 8. Step 1006 can be any embodiment of Step 806illustrated in FIG. 8, Step 1008 can be any embodiment of Step 808illustrated in FIG. 8, and Step 1012 can be any embodiment of Step 816illustrated in FIG. 8. In some embodiments, the method 1000 illustratedin FIG. 10A differs from the method 800 illustrated in FIG. 8 in thatthe method 1000 illustrated in FIG. 10A maintains the client IP addressand the client port.

The packet engine 548 carrying out the steps of the method 1000described in FIG. 10A can execute on a particular core 505. The core505, in most embodiments, is selected ahead of time by the method 1000illustrated in FIG. 10A. Therefore in most instances, the data packetreceived by the packet engine 548 has been allocated to the core 505based on the application of an above-described hash to a tuple of thedata packet. This tuple, in most cases, comprises at least a client IPaddress, a destination IP address, a client port and a destination port.In some embodiments, the tuple can be any of the above described tuplesand can comprise any number of source or destination identifying values.In still other embodiments, the client IP address can be a source IPaddress identifying the machine from which the data packet originated.Similarly, the client port can be a source port.

In one embodiment, a packet engine 548 executing on a particular core505 in the multi-core system 545, receives data packet allocated to thatparticular core 505 (Step 1008). The packet engine 548 can directlyreceive data packets, or in some embodiments, a communication moduleexecuting on the core 505 can receive and transmit data packets. Inother embodiments, a virtual NIC (Not Shown) executing on the core 505can receive and transmit data packets. Receiving data packets, in someembodiments, can further comprise draining data packets from a logicalreceive queue on the core 505. A logical receive queue can store datapackets transmitted to a core 505. The packet engine 548 can access datapackets in the logical receive queue by draining or otherwise obtainingthe data packets from the receive queue according to afirst-in-first-out method of access. Another possible method of accesscan be first-in-last-out. In some embodiments, the packet engine 548executes on a first core 505 and receives the data packets from a flowdistributor based on a hash of a first tuple of the data packets, thefirst tuple comprising a client IP address, a destination IP address, aclient port and a destination port. The data packets, in someembodiments, can be a client request or a server response.

When a packet engine 548 obtains a data packet, the packet engine 548can in some embodiments determine whether the data packet can bemodified. The packet engine 548, after determining what portions of thedata packet can be modified, can modify the data packet. In someembodiments, the multi-core system 545 can be configured to instructpacket engines 548 executing within the multi-core system 545 to modifyonly certain portions of the data packet.

In some embodiments, the packet engine 548 can determine that the datapacket cannot be modified. In other embodiments, the multi-core system545 can be configured such that the data packet is not modified, butrather each element of the tuple of the data packet is maintained. Instill other embodiments, the packet engine 548 is configured responsiveto a security policy of either the first core 505 or the multi-coresystem 545, where the security policy dictates that the client port andthe client IP address are to be maintained. Thus, when the packet engine548 receives the data packet, the packet engine 548 maintains both theclient IP address and the client port, i.e. the source IP address andthe source port (Step 1010). Therefore the packet engine 548 forwards orotherwise transmits the data packet to a remote computing machine orappliance (Step 1012). The data packet, when transmitted, retains atuple comprising the following elements: client IP address; destinationIP address; client port; and destination port.

Transmitting the modified data packet to a remote computing machine cancomprise transmitting the modified data packet to a client, server,appliance, or computing machine identified by the destination IP addressand/or the destination port. In some embodiments, the modified datapacket is transmitted to a proxy server or appliance before the datapacket is transmitted to its destination computing machine or appliance.In other embodiments, the modified data packet is stored in a memoryelement within the multi-core system 545 before the data packet istransmitted to its destination computing machine or appliance. Thememory element, in some embodiments, can be a global cache or othermemory element shared by all cores and devices in the multi-core system545. In other embodiments, the memory element can be a cache or otherstorage repository accessible by the current core 505.

Illustrated in FIG. 10B is a more detailed embodiment of at least oneportion of the method 1000 illustrated in FIG. 10A. The method 1050illustrated in FIG. 10B illustrates an embodiment of the process carriedout once a packet engine 548 on a selected core 505 receives a forwardeddata packet. Upon receiving the data packet (Step 1052), the packetengine 548 can identify a tuple of the data packet and apply theabove-described hash to the identified tuple (Step 1054). After applyingthe hash, the packet engine determines whether the data packet waspreviously handled by the core (Step 1058). When a determination is madethat the data packet was previously handled by the core 505, the packetengine 548 proceeds to process the data packet (Step 1060). When adetermination is made that the data packet was previously handled byanother core 505, the correct destination core 505 is identified via thehash result (Step 1062) and the data packet is forwarded to the correctdestination core (Step 1064).

Further referring to FIG. 10B, and in more detail, in one embodiment themethod 1050 can be carried out by a packet engine 548 on a selected core505. In other embodiments, the method 1050 can be carried out by aninstance of a flow distributor 550, or by any other flow distributionmodule executing on the selected core 505. In some embodiments, theselected core 505 is a core selected by a flow distributor 550 or RSSmodule 560 executing in the multi-core system 545, based on a hash of atuple of the data packet. Therefore, when the multi-core system 545first receives a data packet, the flow distributor 550 or RSS module 560applies any of the above-mentioned hashes to a tuple of the data packet.A result of the hash identifies a core 505 in the multi-core system 545,and the flow distributor 550 or the RSS module 560 forwards the datapacket to the selected core 505. Any reference to a selected core 505 ora present core 505 is in most embodiments a reference to the core 505selected by the flow distributor 550 or RSS module 560 based on a tupleassociated with the data packet.

In one embodiment, a packet engine 548 receives a data packet (Step1052) forwarded to the selected core 505 by a flow distributor 550, RSSmodule 560 or any other flow distribution module. The packet engine 548can directly receive data packets, or in some embodiments, acommunication module executing on the core 505 can receive and transmitdata packets. In other embodiments, a virtual NIC (Not Shown) executingon the core 505 can receive and transmit data packets. Receiving datapackets, in some embodiments, can further comprise draining data packetsfrom a logical receive queue on the core 505. A logical receive queuecan store data packets transmitted to a core 505. The packet engine 548can access data packets in the logical receive queue by draining orotherwise obtaining the data packets from the receive queue according toa first-in-first-out method of access. Another possible method of accesscan be first-in-last-out. In some embodiments, the packet engine 548,executing on a second core 505, can receive the data packets from a flowdistributor 550 based on a hash of a second tuple of the data packets,the second tuple comprising the client IP address, the client port, thedestination IP address and the destination port. The data packets, insome embodiments, can be a server response to a client requestpreviously handled by a first core 505 in the multi-core system 545. Insome embodiments, FIG. 10A illustrates the handling of the clientrequest by the first core 505.

In some embodiments the packet engine 548 applies a hash, such as any ofthe above-described hashes, to a tuple associated with the received datapacket (Step 1054). Applying the hash can further comprise firstidentifying a tuple of the data packet. Determining a tuple of the datapacket can include identifying and concatenating the following values: aclient IP address; a destination IP address; a client port; and adestination port. In one embodiment, the tuple comprises theconcatenation of these values. In some embodiments, the packet engine548 carries out this concatenation, while in other embodiments the tupleis included within the received data packet.

The result of the hash, in some embodiments, identifies a destinationcore 505. This core 505, in some embodiments, identifies the current orselected core 505, while in other embodiments this result identifies acore 505 different from the current or selected core 505. While FIG. 10Billustrates a method 1050 that includes Step 1054, in some embodimentsthe method 1050 does not include Step 1054. In these embodiments, adetermination as to whether the data packet was previously handled bythe current core 505 can be made by comparing attributes of the datapacket with a table or list accessible by the packet engine 548 on thecurrent core, with attributes of the data packet. These attributes canbe any one of a client IP address, a client port, a destination IPaddress, a destination port, a flag stored in metadata, a markingindicating the previous core 505 that handled the data packet or anyother attribute that can be stored in a table or list and used toidentify whether a particular core 505 handled the data packet. Thistable or list can be updated by a packet engine 548 each time the core505 handles a data packet. The update can comprise an entry indicatingthat a data packet having certain characteristics was handled by thecore 505.

The packet engine 548 can review either the resultant hash value or atable tracking packet attributes, to determine whether the current core505 previously handled the current data packet. When the packet engine548 determines that the packet was previously handled by the currentcore 505, the packet engine 548 continues to process the data packet(Step 1060). When the packet engine 548 determines that the packet wasnot previously handled by the current core 505, the packet engine 548identifies the correct core 505 (Step 1062) and forwards the data packetto the correct core (Step 1064).

Determining the correct core 505 (Step 1062), in some embodiments,comprises either reviewing the result of a hash applied to a tuple ofthe data packet (Step 1054). This hash result can be stored in cache oranother memory element or location accessible by a first, second orthird core 505, so that a later determination can be made as to where totransmit a misdirected data packet. In some embodiments, the datapackets can be stored in the cache or other memory element, memorylocation or shared buffer, where this shared buffer is accessible byeach of the cores in the multi-core system 545 including the first coreand the second core. In embodiments where a hash was not previouslyapplied, the packet engine 548 can apply the above-described hash to atuple of the data packet to obtain a resultant hash value. Thisresultant hash value identifies a core 505 in the multi-core system 545that is different from the current or selected core 505. The hashapplied to the second tuple can be the same hash function as the hashfunction applied to the client request, supra.

Determining that the different core, or first core 505, is the correctcore can include determining that the received response corresponds to aclient request not processed by the second packet engine 548 on thesecond core 505. The packet engine 548 can obtain information about theidentified core 505 and forward the data packet to the correctdestination core 505 identified by the hash result (Step 1064). Lookingup information about the correct core 505, or the first core 505 caninclude searching for a port in a port allocation table to identify thefirst core 505. In some embodiments, the packet engine 548 executing onthe second core 505 can send a message to the first core 505 oridentified core 505, indicating that the data packets (i.e. the serverresponse) is to be processed by a packet engine 548 on the first core505.

Forwarding the data packet to the correct destination core 505 (Step1064) can occur one of two ways: either the data packet can be copiedinto a common cache or memory element accessible by both the currentcore 505 and the correct core 505, and the data packet can be downloadedby the correct core 505; or the data packet can be transmitted to thecorrect core 505 via an internal network over which the cores 505communicate with one another. In embodiments where the data packet isstored to a common memory element, the packet engine 548 copies the datapacket into the common cache or common memory element, and sends amessage to a packet engine on the correct core to download the copieddata packet. A core-to-core messaging system or intra-systemcommunication network can be used by the packet engine 548 of thecurrent core 505 to send a message to the packet engine 548 of thecorrect core 505 that instructs the packet engine 548 of the other core505 to download the copied data packet from the shared cache or memoryelement. In embodiments where the data packets are transmitted to thecorrect core 505 via an internal network, the packet engine 548 of thepresent core 505 obtains an address of the packet engine 548 of thecorrect core and forwards the data packet to that address over aninternal network in the multi-core system 545. In some embodiments, thepacket engine of the present core forwards the data packet to a controlcore in the multi-core system 545 which then forwards the data packet tothe correct core. In other embodiments, the packet engine of the presentcore forwards the data packet to a neighboring core which determinesthat it is not the correct core and forwards the data packet to aneighboring core. This process continues until the correct core receivesthe data packet.

5. System and Method for Packet Fragment Steering and Re-Assembly in aMulti-Core Environment

In some embodiments, either a client request, a server response oranother type of data packet can be fragmented. In a multi-core system545 there is an added layer of complexity to reassembling a fragmenteddata packet because the fragmented data packet, in some embodiments, isreceived by a packet engine 548 or flow distributor 550 executing on acore 505 that is not the ultimate destination core 505 for that request,response or data packet. Therefore, the packet engine 548 or flowdistributor 550 must forward either a reassembled data packet or thedata packet fragments to the destination core 505. This destination core505 cannot be determined until at least a port of the data packet headeris reassembled so that the following values can be obtained: a source IPaddress; a destination IP address; a source port; and a destinationport. Once these values are obtained, the packet engine 548 or flowdistributor 550 can forward either a reassembled data packet or the datapacket fragments to a core 505 identified by a hash of theabove-mentioned values.

Illustrated in FIG. 11A is an embodiment of a method 1100 fordistributing fragmented network traffic over one or more cores 505 in amulti-core system 545. The multi-core system 545 receives data packetfragments (Step 1102) and a flow distributor 550 or RSS module 560executing within the multi-core system 545 assembles data packetfragments into the whole data packet until a packet header is reached(Step 1104). Once the header is reached, a tuple comprising a source IPaddress, a destination IP address, a source port and a destination portis identified within the header. The flow distributor 550 or RSS module560 applies a hash to the tuple and the resultant value identifies atleast one core 505 in the multi-core system 545. After identifying thecore 505, the data packet fragments are transmitted to the selected core505 (Step 1106). A packet engine 548 on the selected core 505 receivesthe data packet fragments and forwards them to a fragmentation module650 executing on the selected core 505 (Step 1108). Once thefragmentation module 650 receives the data packet fragments, thefragmentation module 650 reassembles the data packet from the datapacket fragments (Step 1110).

Further referring to FIG. 11A, and in more detail, in one embodiment themethod 1100 can be carried out by a packet engine 548 executing on acore 505. In another embodiment, the method 1100 can be carried out by aflow distributor 550 or an instance of a flow distributor executing onthe core 505. In still other embodiments, the method 1100 can be carriedout by any flow distribution module or agent that may execute on thecore 505. While FIG. 11A contemplates reassembling a data packet fromdata packet fragments, reassembly of the data packet can, in someembodiments, be handled by a control core in the multi-core system 545.

The packet engine 548 carrying out at least a portion of the steps ofthe method 1100 described in FIG. 11A can execute on a particular core505. The core 505, in most embodiments, is selected ahead of time byapplying a hash to a tuple of the data packet fragments. This tuple, inmost cases, comprises at least a client IP address, a destination IPaddress, a client port and a destination port. In some embodiments, thetuple can be any of the above described tuples and can comprise anynumber of source or destination identifying values. In still otherembodiments, the client IP address can be a source IP addressidentifying the machine from which the data packet originated.Similarly, the client port can be a source port.

In one embodiment, a flow distributor 550 executing within themulti-core system 545, receives data packet fragments from a computingmachine or appliance remotely located outside of the multi-core system545 (Step 1102). The flow distributor 550 can directly receive datapacket fragments, or in some embodiments, a communication module canreceive and transmit data packets or data packet fragments. In otherembodiments, the NIC 552 can receive and transmit data packets and datapacket fragments. Receiving data packets and data packet fragments, insome embodiments, can further comprise draining data packets or datapacket fragments from a receive queue on the NIC 552. A receive queuecan store data packets and data packet fragments transmitted to themulti-core system 545. The flow distributor 550 can access data packetsand data packet fragments in the receive queue by draining or otherwiseobtaining the data packets and data packet fragments from the receivequeue according to a first-in-first-out method of access. Anotherpossible method of access can be first-in-last-out.

In some embodiments, a packet engine 548 can receive a client requestthat identifies a first tuple comprising a client internet protocoladdress, a client port, a server internet protocol address and a serverport. In these embodiments, the packet engine 548 can execute on a core505 selected by the flow distributor 550 based on a hash of the firsttuple. The flow distributor 550 can then receive a plurality offragments of a response (Step 1102) to the client request received bythe packet engine 548, the fragments of a response sent by a serverresponsive to receiving the client request that was forwarded by thepacket engine 548 executing on the core 505.

Once the flow distributor 550 receives one or more data packet fragments(Step 1102), the flow distributor 550 can begin to reassemble the datapacket from the data packet fragments until a packet header is reached(Step 1104). In some embodiments, the entire data packet is reassembledby the flow distributor 550 from the received data packet fragments. Inother embodiments, only those portions of the data packet that make upthe header are assembled by the flow distributor 550. In still otherembodiments, the flow distributor 550 can begin to reassemble the datapacket from the data packet fragments until the flow distributor 550 isable to extract from the partially assembled data packet the followinginformation: a source IP address; a destination IP address; a sourceport; and a destination port. This information, in many embodiments, isstored in the packet header. Thus, the flow distributor 550 ceasesreassembling the data packet from the data packet fragments when theflow distributor 550 determines that at least a portion of the partiallyreassembled data packet comprises a data packet header. Determining thatat least a portion of the partially reassembled data packet comprises adata packet header can comprise assembling a portion of the plurality offragments, and/or assembling the portion of the plurality of fragmentsuntil a header of the response is assembled.

Once a header has been identified, the flow distributor 550 can identifya tuple (i.e. a second tuple, third tuple, or first tuple) of the datapacket, where the tuple can be any tuple described herein. The tuple, insome embodiments, comprises a concatenation or string of the followingvalues extracted from the data packet header: a source IP address; adestination IP address; a source port; and a destination port. In otherembodiments, the tuple can comprise at least a source IP address and adestination IP address identified by the plurality of fragments.Identifying the tuple can further include extracting from the datapacket header or the response header any of the tuple contents (i.e. thesource IP address, and the destination IP address.) Once the tuple isidentified, the flow distributor 550 applies the above-described hash tothe identified tuple to generate a second, third or first hash. Theresult of the hash identifies a core 505 (i.e. a second core) in themulti-core system 545. This identified core 505 can be referred to asthe destination core 505 or the second core 505. The flow distributor550, or any other communication module within the multi-core system 545,transmits the data packet fragments to the destination core 505 (Step1106).

A packet engine 548 executing on the destination core 505 can receivethe data packet fragments and can forward the data packet fragments to afragmentation module 650 executing on the destination core 505 (Step1108). In some embodiments, the packet engine 548 can store theplurality of fragments upon receiving them. The packet engine can storethe plurality of fragments in a memory location or cache accessible bythe core 505 that originally received the fragments and the destinationcore 505. Upon receiving the data packet fragments, the fragmentationmodule 650 reassembles the data packet from the received data packetfragments (Step 1110). In some embodiments, rather than permitting afragmentation module 650 to reassemble the data packet, the packetengine 548 performs on the plurality of fragments a fragmentationaction, and determines by a rule of the flow distributor executing onthe destination core 505, to direct the plurality of fragments receivedby the destination core 505 to the first core 505 or the core thatinitially received the request. In these embodiments, the fragmentationaction can be an assemble action directing the packet engine 548 or afragmentation module 650 to reassemble the data packets, or can be abridging action directing the packet engine 548 or a fragmentationmodule 650 to transmit or steer the data packets to a first core 505 oranother core 505 (i.e. a second core 505, a third core 505.) Determiningthat the plurality of fragments should be transmitted from thedestination core 505 to the first core 505 can, in some embodiments,include first determining that the first core 505 handled the clientrequest or otherwise established a connection between the client and theserver. Directing the plurality of fragments from the destination core505 to the first core 505 can further include sending, by a packetengine 548 on the destination core 505, a packet engine 548 on the firstcore 50 a message directing the packet engine 548 on the first core 505to process the assembled plurality of fragments.

While the above-mentioned method 1100 is partially carried out by a flowdistributor 550, those steps carried out by the flow distributor 550 canbe carried out by a packet engine 548 executing on a first core 505A. Insome embodiments, data packet fragments can be forwarded to a defaultcore dedicated to handling data packet fragments. Rather than processthe data packet fragments using the flow distributor 550 or RSS module560, the system can be configured to forward all data packet fragmentsto a first core 505A having a fragmentation module 650 or an instance ofa fragmentation module 650 executing thereon. This fragmentation module650 can reassemble a data packet until the relevant portions of the datapacket are available for extraction by a flow distributor instance 550executing on the default core.

When a packet engine 548 executing on a default core or first core 505Areceives the fragmented data packet, the packet engine 548 can transmitthe data packet fragments to a destination core via a core-to-coremessaging system, or via an intra-multi-core system communicationnetwork. In some embodiments, transmitting the data packet fragments(Step 1106) can comprise copying the data packet fragments into a globalcache or memory element, and sending a message to a destination core orpacket engine executing on the destination core instructing the packetengine to download the data packet fragments from global cache. In otherembodiments, the data packet fragments can be encapsulated withinanother packet header indicating that the data packet fragments shouldbe transmitted to the packet engine 548 of the destination core 505.These data packet fragments can be sent to the destination packet engineover an internal network in the multi-core system 545.

In other embodiments, the above-mentioned method 1100 can be carried outby a flow distributor 550 or RSS module 560 further executing or havinga fragmentation module. The fragmentation module can handle all datapacket fragments intercepted or received by the flow distributor 550 orRSS module 560.

Illustrated in FIG. 11B is another embodiment of a method 1150 forallocating or distributing data packet fragments to cores 505 in amulti-core system 545. A flow distributor 550 or RSS module 560 receivesdata packet fragments (Step 1152), and assembles a data packet from thedata packet fragments until a packet header is reached (Step 1154). Oncethe header is reassembled, the flow distributor 550 or RSS module 560can extract the following values to create a tuple or string of thosevalues, the values are: a source IP address; a destination IP address; asource port; and a destination port. After creating or identifying atuple of the reassembled header, a hash is applied to the tuple. In mostembodiments, the hash result identifies a core in the multi-core system545 (Step 1156), this core can be referred to as a destination core.Once a destination core 505 is identified, a fragmentation action can bedetermined (Step 1158). If the fragmentation action is “Assemble,” (Step1160) then a data packet is reassembled from the data packet fragments(Step 1164) and the reassembled data packet can be transmitted to apacket engine on the destination core (Step 1166). When thefragmentation action is not “Assemble,” then the data packet fragmentscan be steered to a destination packet engine executing on thedestination core 505 (Step 1162).

Further referring to FIG. 11B, and in more detail, in one embodiment themethod 1150 can be carried out by a packet engine 548 executing on acore 505. In another embodiment, the method 1150 can be carried out by aflow distributor 550 or an instance of a flow distributor executing onthe core 505. In still other embodiments, the method 1100 can be carriedout by any flow distribution module or agent that may execute on thecore 505. While FIG. 11B contemplates reassembling a data packet fromdata packet fragments, reassembly of the data packet can, in someembodiments, be handled by a control core in the multi-core system 545.

The packet engine 548 carrying out at least a portion of the steps ofthe method 1150 described in FIG. 11B can execute on a particular core505. The core 505, in most embodiments, is selected ahead of time byapplying a hash to a tuple of the data packet fragments. This tuple, inmost cases, comprises at least a client IP address, a destination IPaddress, a client port and a destination port. In some embodiments, thetuple can be any of the above described tuples and can comprise anynumber of source or destination identifying values. In still otherembodiments, the client IP address can be a source IP addressidentifying the machine from which the data packet originated.Similarly, the client port can be a source port.

In one embodiment, a flow distributor 550 executing within themulti-core system 545, receives data packet fragments from a computingmachine or appliance remotely located outside of the multi-core system545 (Step 1152). The flow distributor 550 can directly receive datapacket fragments, or in some embodiments, a communication module canreceive and transmit data packets or data packet fragments. In otherembodiments, the NIC 552 can receive and transmit data packets and datapacket fragments. Receiving data packets and data packet fragments, insome embodiments, can further comprise draining data packets or datapacket fragments from a receive queue on the NIC 552. A receive queuecan store data packets and data packet fragments transmitted to themulti-core system 545. The flow distributor 550 can access data packetsand data packet fragments in the receive queue by draining or otherwiseobtaining the data packets and data packet fragments from the receivequeue according to a first-in-first-out method of access. Anotherpossible method of access can be first-in-last-out.

In some embodiments, a packet engine 548 can receive a client requestthat identifies a first tuple comprising a client internet protocoladdress, a client port, a server internet protocol address and a serverport. In these embodiments, the packet engine 548 can execute on a core505 selected by the flow distributor 550 based on a hash of the firsttuple. The flow distributor 550 can then receive a plurality offragments of a response (Step 1102) to the client request received bythe packet engine 548, the fragments of a response sent by a serverresponsive to receiving the client request that was forwarded by thepacket engine 548 executing on the core 505.

Once the flow distributor 550 receives one or more data packet fragments(Step 1152), the flow distributor 550 can begin to reassemble the datapacket from the data packet fragments until a packet header is reached(Step 1154). In some embodiments, the entire data packet is reassembledby the flow distributor 550 from the received data packet fragments. Inother embodiments, only those portions of the data packet that make upthe header are assembled by the flow distributor 550. In still otherembodiments, the flow distributor 550 can begin to reassemble the datapacket from the data packet fragments until the flow distributor 550 isable to extract from the partially assembled data packet the followinginformation: a source IP address; a destination IP address; a sourceport; and a destination port. This information, in many embodiments, isstored in the packet header. Thus, the flow distributor 550 ceasesreassembling the data packet from the data packet fragments when theflow distributor 550 determines that at least a portion of the partiallyreassembled data packet comprises a data packet header. Determining thatat least a portion of the partially reassembled data packet comprises adata packet header can comprise assembling a portion of the plurality offragments, and/or assembling the portion of the plurality of fragmentsuntil a header of the response is assembled.

Once a header has been identified, the flow distributor 550 can identifya tuple (i.e. a first tuple, a second tuple, a third tuple) of the datapacket, where the tuple can be any tuple described herein. The tuple, insome embodiments, comprises a concatenation or string of the followingvalues extracted from the data packet header: a source IP address; adestination IP address; a source port; and a destination port. Once thetuple is identified, the flow distributor 550 applies theabove-described hash to the identified tuple to generate a second, thirdor first hash. The result of the hash identifies a core 505 (i.e. asecond core) in the multi-core system 545. This identified core 505 canbe referred to as the destination core 505 or the second core 505. Theflow distributor 550, or any other communication module within themulti-core system 545, transmits the data packet fragments to thedestination core 505 (Step 1156).

The flow distributor 550 can then determine a fragmentation actionassociated with the data packet fragments (Step 1158). In someembodiments, the fragmentation action is dictated by the multi-coresystem 545. An administrator can configure the multi-core system 545 toeither “Bridge” the data packet fragments to a destination core bytransmitting each data packet fragment to the destination core where thefragments are reassembled. In other embodiments, the administrator canconfigure the multi-core system 545 to “Assemble” the data packetfragments into the data packet prior to transmitting the data packet tothe destination core. In other embodiments, the fragmentation action canbe identified in the data packet header or in metadata associated witheach data packet. In still other embodiments, the decision whether to“Assemble” or “Bridge” can be made based on any combination of thefollowing criteria: the number of data packet fragments; the type ofdata within the data packet load; the size of each data packet fragment;the size of the data packet; the source IP address; the destination IPaddress; the amount of available processing resources in the multi-coresystem 545; or any other factor. In embodiments where the flowdistributor 550 takes into account data packet size, the flowdistributor 550 may “Assemble” data packets when it is determined thatthe data packet size is too great to transmit piecemeal according to the“Bridge” fragmentation action. When the flow distributor 550 takes intoaccount the amount of available processing resources, the flowdistributor 550 may analyze the amount of load on the destination coreand determine whether the destination core has enough availableresources to assemble the data packet. In some embodiments, the decisionwhether to “Assemble” or “Bridge” the data packet fragments can be basedon a determination as to whether the destination core has afragmentation module 650. In embodiments where the destination core hasa fragmentation module 650, the data packet fragments are “Bridged.” Inembodiments where the destination core does not have a fragmentationmodule 650, the data packet fragments are “Assembled.”

When, in some embodiments, the fragmentation action is “Assemble” (Step1160), the data packets are reassembled by the flow distributor 550 orby a fragmentation module executing within the flow distributor, intothe data packet (Step 1164). Once the data packet is reassembled fromthe data packet fragments, the data packet is transmitted to thedestination core where it is received by a packet engine executing onthe destination core (Step 1166). In some embodiments, the data packetfragments are stored in a fragmentation table 655 prior to transmittingthe reassembled data packet to the destination core.

When, in some embodiments, the fragmentation action is “Bridge” (Step1160), the data packets are steered to the destination core where theyare reassembled (Step 1162). In some embodiments, a packet engineexecuting on the destination core receives the data packet fragments andeither assembles them, or transmits them to a fragmentation module 650where they are reassembled. In some embodiments, the data packetfragments are stored in a fragmentation table 655 prior to transmittingeach data packet fragment to the destination core. In other embodiments,the packet engine 548 can store the plurality of fragments uponreceiving them. The packet engine can store the plurality of fragmentsin a memory location or cache accessible by the core 505 that originallyreceived the fragments and the destination core 505. The data packetfragments, in some embodiments are transmitted or steered to thedestination core in the order in which they were received by the client,server or other computing machine or appliance.

In embodiments where a data packet has a TCP header and any of thefollowing happens, the fragmentation action is “Assemble”: the traffichits a PCB; the traffic hits NATPCB and an “Assemble Packet” flag isset; the traffic hits a configured service or packet engine whose typeis not UDP; and any RNAT traffic. If any of this does not occur, thenthe fragmentation action is “Bridge.” In embodiments where a data packethas a UDP header any of the following happens, the fragmentation actionis “Assemble”: the traffic hits NATPCB and the “Assemble Packet” flag isset; the traffic hit a configured service or packet engine whose type isnot UDP. If any of this does not occur, then the fragmentation action is“Bridge.”

The fragmentation action, in some embodiments, can be determined bydoing service, RNAT, PCB and NATPCB lookups. Service and RNAT lookupscan, in some embodiments, be done on any packet engine. However, thePCB/NATPCB that manages the connection may not reside in the same packetengine as a packet engine that receives the fragments.

While the above-mentioned method 1150 is partially carried out by a flowdistributor 550, those steps carried out by the flow distributor 550 canbe carried out by a packet engine 548 executing on a first core 505A. Insome embodiments, data packet fragments can be forwarded to a defaultcore dedicated to handling data packet fragments. Rather than processthe data packet fragments using the flow distributor 550 or RSS module560, the system can be configured to forward all data packet fragmentsto a first core 505A having a fragmentation module 650 or an instance ofa fragmentation module 650 executing thereon. The packet engine 548, inconjunction with the fragmentation module 650, can either reassembledata packets from data packet fragments or steer the data packetfragments to a destination core.

When a packet engine 548 executing on a default core or first core 505Areceives the fragmented data packet, the packet engine 548 can transmiteither the data packet fragments or the reassembled data packet to adestination core via a core-to-core messaging system, or via anintra-multi-core system communication network. In some embodiments,transmitting the data packet fragments or data packet can comprisecopying the data packet fragments or data packet into a global cache ormemory element, and sending a message to a destination core or packetengine executing on the destination core instructing the packet engineto download the data packet or data packet fragments from global cache.In other embodiments, the data packet or data packet fragments can beencapsulated within another packet header indicating that the datapacket fragments should be transmitted to the packet engine 548 of thedestination core 505. These data packet fragments can be sent to thedestination packet engine over an internal network in the multi-coresystem 545.

In other embodiments, the above-mentioned method 1150 can be carried outby a flow distributor 550 or a RSS module 560 further executing orhaving a fragmentation module. The fragmentation module can handle alldata packet fragments intercepted or received by the flow distributor550 or RSS module 560.

What is claimed is:
 1. A method comprising: (a) receiving, by a core ofa device comprising a plurality of cores, a packet of a request of aclient to access a server, the packet comprising a source internetprotocol (IP) address of the client, a source port of the client, adestination IP address of the server, and a destination port of theserver, the core selected to receive the packet based on a first hash ofthe source IP address, the source port, the destination IP address andthe destination port; (b) determining, by the core, that the source IPaddress of the client is to be maintained when forwarding the packet tothe server; (c) selecting, by the core responsive to the determination,a port number from a plurality of port numbers in a port allocationtable of the first core by iterating through one or more of theplurality of port numbers until a second hash of the source IP address,the selected port number, the destination IP address and the destinationport identifies the core for receiving a response to the request fromthe server; and (d) modifying, by the core, the source port of thepacket to be the selected port number; and (e) forwarding, by thedevice, the modified packet to the server.
 2. The method of claim 1,further comprising maintaining, by the device, a plurality of portallocation tables for each core.
 3. The method of claim 2, wherein (c)further comprises selecting, by the core, the port allocation table fromthe plurality of port allocation tables.
 4. The method of claim 1,wherein (c) further comprises selecting a first port number from theplurality of port numbers available in the port allocation table anddetermining that a third hash of the source IP address, the first portnumber, the destination IP address and the destination port does notidentify the core.
 5. The method of claim 4, further comprisingselecting the port number as a second port number from the plurality ofport numbers available in the port allocation table and determining thatthe second hash of the source IP address, the port number, thedestination IP address and the destination port identifies the core. 6.The method of claim 4, further comprising determining that the portnumber is one of available or not in use by the core.
 7. The method ofclaim 1, wherein (c) further comprises iterating through the pluralityof port numbers in order of numbering.
 8. The method of claim 1, wherein(c) further comprises iterating through the plurality of port numbers inorder of listing in the port allocation table.
 9. The method of claim 1,wherein modifying of (d) further comprises replacing the client port inthe source port of the packet with the selected port number.
 10. Themethod of claim 1, further comprising receiving, by the core, theresponse from the server and modifying the destination port thatidentifies the selected port number of the device with the client portand forwarding the modified response to the client.
 11. A systemcomprising: a device comprising a plurality of cores; a core of theplurality of cores configured to receive, a packet of a request of aclient to access a server, wherein the packet comprises a sourceinternet protocol (IP) address of the client, a source port of theclient, a destination IP address of the server, and a destination portof the server, wherein the core was selected to receive the packet basedon a first hash of the source IP address, the source port, thedestination IP address and the destination port; wherein the core isconfigured to determine that the source IP address of the client is tobe maintained when forwarding the packet to the server; wherein the coreis configured to select, responsive to the determination, a port numberfrom a plurality of port numbers in a port allocation table of the firstcore by iterating through one or more of the plurality of port numbersuntil a second hash of the source IP address, the selected port number,the destination IP address and the destination port identifies the corefor receiving a response to the request from the server; wherein thecore is configured to modify the source port of the packet to be theselected port number; and wherein the device is configured to forwardthe modified packet to the server.
 12. The system of claim 11, whereinthe device is configured to maintain a plurality of port allocationtables for each core.
 13. The system of claim 12, wherein the core isconfigured to select the port allocation table from the plurality ofport allocation tables.
 14. The system of claim 11, wherein the core isconfigured to select a first port number from the plurality of portnumbers available in the port allocation table and determines that athird hash of the source IP address, the first port number, thedestination IP address and the destination port does not identify thecore.
 15. The system of claim 14, wherein the core is configured toselect the port number as a second port number from the plurality ofport numbers available in the port allocation table and determines thatthe second hash of the source IP address, the port number, thedestination IP address and the destination port identifies the core. 16.The system of claim 14, wherein the core is configured to determine thatthe port number is one of available or not in use by the core.
 17. Thesystem of claim 11, wherein the core is configured to iterate throughthe plurality of port numbers in order of numbering.
 18. The system ofclaim 11, wherein the core is configured to iterate through theplurality of port numbers in order of listing in the port allocationtable.
 19. The system of claim 11, wherein the core is configured tomodify the packet by replacing the client port in the source port of thepacket with the selected port number.
 20. The system of claim 1, whereinthe core is configured to receive the response from the server andmodify the destination port that identifies the selected port number ofthe device with the client port and forward the modified response to theclient.